Systems and methods for guiding tissue resection

ABSTRACT

A method for guiding resection of local tissue from a patient includes generating at least one image of the patient, automatically determining a plurality of surgical guidance cues indicating three-dimensional spatial properties associated with the local tissue, and generating a visualization of the surgical guidance cues relative to the surface. A system for generating surgical guidance cues for resection of a local tissue from a patient includes a location module for processing at least one image of the patient to determine three-dimensional spatial properties of the local tissue, and a surgical cue generator for generating the surgical guidance cues based upon the three-dimensional spatial properties. A patient-specific locator form for guiding resection of local tissue from a patient includes a locator form surface matching surface of the patient, and a plurality of features indicating a plurality of surgical guidance cues, respectively.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/735,907 with a § 371 date of Dec. 12, 2017, which is a 35U.S.C. § 371 filing of International Application No. PCT/US2016/037043,filed Jun. 10, 2016, which claims priority to U.S. patent applicationSer. No. 14/919,411 filed on Oct. 21, 2015, to U.S. Provisional PatentApplication Ser. No. 62/185,292 filed on Jun. 26, 2015, and to U.S.Provisional Patent Application Ser. No. 62/174,949 filed on Jun. 12,2015. U.S. patent application Ser. No. 14/919,411 is also acontinuation-in-part of U.S. patent application Ser. No. 14/000,068filed on Oct. 22, 2013, which is a 35 U.S.C. § 371 filing ofInternational Patent Application Serial No. PCT/US2012/025671 filed onFeb. 17, 2012, which claims priority to U.S. Provisional PatentApplication Ser. No. 61/443,793 filed on Feb. 17, 2011. All of theabove-identified applications are incorporated herein by reference intheir entireties.

U.S. GOVERNMENT RIGHTS

This invention was made with Government support under NIH Grant No.R21CA182956 awarded by the National Cancer Institute under the NationalInstitutes of Health. The Government has certain rights in thisinvention.

BACKGROUND

Tissue removal surgery is frequently assisted by navigation technologyto guide the surgical procedure in real time. For example, a biopsy maybe guided by ultrasound imaging to ensure that the biopsy is performedat the required location, or removal of the intervertebral disc from aspinal segment may be guided by fluoroscopic x-ray guidance to avoiddamaging the spinal cord or nerve roots. Cancer-related tissue removalprocedures generally require particularly high accuracy. When performinga biopsy of tissue suspected to be cancerous, proper diagnosis relies onsample retrieval from tumor, hence from a specified location, not fromnearby normal tissues. When surgically removing a tumor, any canceroustissue inadvertently left behind may be detrimental to the patient.

Traditionally, breast tumor resection is guided by a wire penetratingthe breast to reach the tumor or a radio-opaque clip placed within thetumor. Breast tumor resection is the removal of the cancerous tissueonly, as opposed to removal of the whole breast. The radio-opaque clipmay be placed in the tumor during a biopsy procedure. The wire insertionis guided by imaging, for example ultrasound imaging, magneto resonanceimaging (MRI), or mammography. It is challenging to ensure that theentire perimeter of the tumor is removed including any filaments orfimbriae. Frequently, since some cancerous tissue remains in the breastafter the resection surgery, breast tumor resection is usuallyaccompanied by radiation treatment with the intent of destroying anyunremoved cancerous tissue. Nevertheless, about one in four women havingundergone breast tumor resection need to return for further resection ofcancerous tissue at or near the site of the original resection.

SUMMARY

In an embodiment, a method for guiding resection of local tissue from apatient includes generating at least one image of the patient, whereinthe at least one image includes an image of the local tissue and animage of at least a portion of surface of the patient. The methodfurther includes automatically determining, at least in part based uponthe at least one image, a plurality of surgical guidance cues indicatingthree-dimensional spatial properties associated with the local tissue,and generating a visualization of the surgical guidance cues relative tothe surface.

In an embodiment, a system for generating surgical guidance cues forresection of a local tissue from a patient includes a location modulefor processing at least one image of the patient to determinethree-dimensional spatial properties of the local tissue, and a surgicalcue generator for generating the surgical guidance cues based upon thethree-dimensional spatial properties.

In an embodiment, a patient-specific locator form for guiding resectionof local tissue from a patient includes a locator form surface thatmatches the surface of the patient near location of the local tissue,such that the patient-specific locator form fits the surface of thepatient near the location of the local tissue. The patient-specificlocator form further includes a plurality of features indicating aplurality of surgical guidance cues, respectively.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features and advantages of the disclosure willbe apparent from the more particular description of embodiments, asillustrated in the accompanying drawings, in which like referencecharacters refer to the same parts throughout the different figures. Thedrawings are not necessarily to scale, emphasis instead being placedupon illustrating the principles of the disclosure.

FIG. 1 illustrates a system for guiding resection of a tumor from abreast of a patient, according to an embodiment.

FIGS. 2A and 2B illustrate exemplary surgical guidance cues for guidingresection of a tumor.

FIG. 2C shows exemplary tumor margins that include a volumetric safetymargin.

FIG. 3 shows the system of FIG. 1 in further detail, according to anembodiment.

FIGS. 4A and 4B illustrate a method for guiding resection of a tumorfrom a breast of a patient, according to an embodiment.

FIG. 5A illustrates a method for generating surgical guidance cues fromat least one supine image, according to an embodiment.

FIGS. 5B and 5C illustrate another method for generating surgicalguidance cues from at least one supine image, according to anembodiment.

FIG. 6 illustrates a method for generating surgical guidance cues fromat least one supine image and a user-defined incision site for tumorresection of tumor, according to an embodiment.

FIG. 7 illustrates a method for generating supine images, wherein thosesupine images include separate volumetric and surface images, accordingto an embodiment.

FIG. 8 illustrates a method for generating at least one supine image,wherein each of the at least one supine image is a volumetric image,according to an embodiment.

FIG. 9 illustrates a method of generating at least one supine imagewhile accounting for tissue displacement of the breast betweenpre-operative imaging and resection surgery, according to an embodiment.

FIGS. 10A and 10B illustrate a method for guiding resection of localtissue from a patient, according to an embodiment.

FIG. 11 illustrates a method for generating at least one imageassociated with a tissue resection procedure, while accounting fortissue displacement between at least a portion of pre-operative imagingand the resection procedure, according to an embodiment.

FIG. 12 illustrates a method for using a rigid-body transformation toregister an initial volumetric image of a patient to a 3D surface imageof the patient substantially as positioned during resection surgery,according to an embodiment.

FIG. 13 illustrates a method for using a rigid-body transformation toregister an initial volumetric image of a patient to a 3D surface imageof the patient substantially as positioned during resection surgery, andfurther using binary image generation and deformable transformationthereof to emphasize certain features, according to an embodiment.

FIG. 14 illustrates a method for generating a binary version of aregistered volumetric image of the method of FIG. 13, according to anembodiment.

FIG. 15 illustrates a finite element modeling method for deformablyregistering a prone volumetric image of a breast to aresection-associated supine 3D surface image of the breast, according toan embodiment.

FIGS. 16A-C and 17A-C illustrate exemplary tissue displacement betweenprone and supine positions of a breast, according to an embodiment.

FIGS. 18A-D show an example of rigid-body transformation of a volumetricimage of a breast, captured with the breast in an initial supineposition, to register the volumetric image to a 3D surface imagecaptured with the breast in the supine position used during resectionsurgery, according to an embodiment.

FIG. 19 illustrates a computer that implements a portion of the systemof FIG. 3, according to an embodiment.

FIG. 20 illustrates a patient-specific locator form for guiding tissueresection, according to an embodiment.

FIG. 21A illustrates functionality of raised needle ports of the locatorform of FIG. 20, according to an embodiment.

FIGS. 21B and 21C illustrate a two-part raised needle port of thelocator form of FIG. 20, which targets two different positions,according to an embodiment.

FIG. 22 illustrates a method for guiding tumor resection using a locatorform, according to an embodiment.

FIG. 23 illustrates a patient-specific locator form that furtherincludes a material model of a tumor, according to an embodiment.

FIG. 24 illustrates a system for manufacturing a patient-specificlocator form, according to an embodiment.

FIG. 25 illustrates a system for manufacturing a plurality ofpatient-specific locator forms for a respective plurality of patients,according to an embodiment.

FIG. 26 illustrates a method for producing a patient-specific locatorform, according to an embodiment.

FIG. 27 illustrates a method for producing a material model of a tumorand a connecting rod for incorporating the material model into apatient-specific locator form, according to an embodiment.

FIG. 28 illustrates a navigation system for guiding resection of a tumorfrom a breast with the aid of surgical guidance cues, according to anembodiment.

FIG. 29 illustrates a method for visualizing surgical guidance cuesusing a navigation system, according to an embodiment.

FIG. 30 illustrates a method for transferring surgical guidance cues toa breast using a navigation system and tracking devices, according to anembodiment.

FIG. 31 illustrates a method for automatically transferring surgicalguidance cues to a breast using a robotic system, according to anembodiment.

FIG. 32 illustrates a method for guiding tissue, which does not requireimage registration, according to an embodiment.

FIG. 33 illustrates a method for guiding tumor resection based uponpreoperative supine volumetric image(s) and supine 3D surface image(s)representative of the supine position used during resection surgery,according to an embodiment.

FIG. 34 illustrates a method for guiding resection of a tumor, whichextracts both volumetric and surface data from the same preoperativevolumetric image(s), according to an embodiment.

FIG. 35 shows exemplary image data illustrating image processing by oneembodiment of the method of FIG. 34.

FIG. 36 illustrates a method for guiding resection of a tumor, using apatient-specific locator form manufactured based upon volumetric andsurface data obtained from the same preoperative volumetric image(s),according to an embodiment.

FIG. 37 illustrates a method for guiding resection of a tumor based uponpreoperative prone volumetric image(s) and supine 3D surface image(s)representative of the supine position used during resection surgery,according to an embodiment.

FIG. 38 illustrates a method for guiding resection of a tumor, whereinsurgical guidance cues are transferred to the breast preoperatively,according to an embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1 illustrates one exemplary system 100 for guiding resection of atumor 175 from a breast 172 of a patient 170. The resection surgery isperformed with patient 170 in supine position, i.e., with patient 170laying on her back and breast 172 facing upwards. System 100 utilizessupine images 158 of breast 172 to determine geometrical properties ofthe resection surgery to remove tumor 175, and also generate surgicalguidance cues 138 to guide the resection surgery. Herein, a “supineimage” refers to an image of patient 170 in the supine position. Eachsupine image 158 is an image of breast 172 corresponding to breast 172being in the supine position and substantially in the position in whichthe resection surgery is performed. Supine image 158 is captured whilepatient 170 is in the supine position or, alternatively, generated froma combination of images captured in the supine and prone positions.Surgical guidance cues 138 directly and/or indirectly indicatethree-dimensional (3D) spatial properties of tumor 175 in the supineposition used during resection surgery.

System 100 includes a location module 120 that processes at least onesupine image 158 of breast 172 to determine 3D spatial properties 128 oftumor 175. System 100 further includes a surgical cue generator 130 thatdetermines surgical guidance cues 138. A surgeon 180 uses surgicalguidance cues 138 to perform resection surgery on patient 170 to removetumor 175. Herein, a “surgeon” may refer to one or more humans, one ormore computer systems, one or more robotic devices, and/or a combinationthereof.

In an embodiment, system 100 further includes a visualization module 140that visualizes surgical guidance cues 138 for surgeon 180. In oneexample, visualization module 140 displays surgical guidance cues 138 ona computer-generated model of breast 172.

In an embodiment, system 100 includes an imaging module 150 thatcaptures the at least one supine image 158, or alternatively capturesone or more images, from which at least one supine image 158 may begenerated. The at least one supine image 158 includes an image of tumor175 and an image of at least a portion of surface 174 of breast 172.Imaging module 150 includes a volumetric imager 152 that captures a 3Dimage of breast 172 including an image of tumor 175. Volumetric imager152 may include a magnetic resonance imaging (MRI) scanner, anultrasound imaging device, a computerized tomography (CT) scanner, amammography X-Ray instrument, and/or another volumetric imaging systemknown in the art. Imaging module 150 may also include a surface imager154 that captures a 3D surface image of at least a portion of surface174. Surface imager 154 may include a stereo camera, a structured-lightimaging device, an optical-scattering device, and/or an optical surfaceimager known in the art. In embodiments where imaging module 150 doesnot include surface imager 154, at least one volumetric image capturedby volumetric imager 152 includes an image of surface 174 or a portionthereof. For example, a portion of surface 174 may be identifiable in amagnetic resonance (MR) image of breast 172.

FIGS. 2A and 2B illustrate exemplary surgical guidance cues 138 forguiding resection of tumor 175. FIG. 2A is a cross sectional, lateralview of a right breast 202. Right breast 202 is an example of breast172. FIG. 2B is an anterior (frontal) view of right breast 202 and anassociated left breast 204. FIGS. 2A and 2B are best viewed together.Breast 202 has a tumor 210. Tumor 210 is an example of tumor 175. Breast202 has a surface 220. Surface 220 is an example of surface 174. FIG. 2Afurther indicates the chest wall 240 at breast 202.

Exemplary surgical guidance cues 138 include (a) the position of point222 which is the point of surface 220 closest to tumor 210, (b) theprojection 224 of tumor 210 onto surface 220 along line-of-sight 230from centroid 212 to point 222, (c) the anterior margin 214 of tumor 175(i.e., point of tumor 210 where line-of-sight 230 intersects theanterior perimeter of tumor 175), which may be indicated as distance 226between anterior margin 214 and point 222, and/or (d) the posteriormargin 216 of tumor 175 (i.e., the point of tumor 210 whereline-of-sight 230 intersects the posterior perimeter of tumor 175),which may be indicated as distance 234 between posterior margin 216 andpoint 232 where line-of-sight 230 intersects chest wall 240.

Optionally, projection 224 is indicated as a set of surgical guidancecues 138 including one or more margins of projection 224 such as theposition of cranial margin 242 (the upwards extreme of projection 224,wherein upwards is in direction of the patient's head), the position ofcaudal margin 244 (the downwards extreme of projection 224, whereindownwards is in direction of the patient's feet), the position oflateral margin 246 (the most lateral point of projection 224), and/orthe position of medial margin 248 (the most medial point of projection224).

In certain examples of use, the definitions of cranial margin 242,caudal margin 244, lateral margin 246, and medial margin 248 incorporatea safety margin, such that each of cranial margin 242, caudal margin244, lateral margin 246, and medial margin 248 is defined as therespective upwards, downwards, lateral, and medial extremes of a bodythat is the tumor plus an additional volumetric safety margin. Theadditional volumetric safety margin is, in some embodiments, about acentimeter in extent, such as between 0.5 and 2.0 centimeters. Theadditional volumetric safety margin may be calculated using a standarduniform mesh re-sampling technique.

FIG. 2C shows exemplary tumor margins that include a volumetric safetymargin. A model 280 of tumor 175 is extended by a volumetric safetymargin 282 with thickness 284. Thickness 284 may be about a centimeterin extent, such as between 0.5 and 2.0 centimeters. Cranial margin 242,caudal margin 244, lateral margin 246, and medial margin 248 are definedas the respective upwards, downwards, lateral, and medial extremes ofsafety margin 282. Safety margin 282 may be truncated by the boundary ofbreast 172 in scenarios where tumor 175 is relatively close to aboundary of breast 172.

Other exemplary surgical guidance cues 138 include the position ofcentroid 212, the outline of tumor 210, and/or the full volumetricextent of tumor 210. Furthermore, surgical guidance cues 138 may includeadditional positions, for example to account for tumors 210 of complexgeometry. In one such example, surgical guidance cues 138 includes otherpositions on the perimeter of tumor 210, or within tumor 210, optionallyin addition to one or more of the surgical guidance cues discussedabove.

In an alternative embodiment, point 222 is defined as the incision pointfor the resection surgery to remove tumor 210 from breast 202. In thisembodiment, point 222 is not necessarily the point of surface 220closest to tumor 210. Point 222 may be a user-defined incision point.For example, surgeon 180 may choose an incision point based at least inpart upon cosmetic considerations, considerations of the surface tissueof breast 202 (for example the presence of scar tissue), or practicalconsiderations such as ease of access to the incision point.

FIG. 3 shows system 100 in further detail. Location module 120 mayinclude one or more of a feature locator 322, a position calculator 324,a direction calculator 326, and a tumor perimeter calculator 328.Feature locator 322 identifies one or more features in supine images158, such as tumor 175 and at least a portion of surface 174. Positioncalculator 324 determines positions such as the position of centroid 212and/or the position of point 222. Direction calculator 326 determinesline-of-sight 230 and/or vector(s) along line-of-sight 230. Tumorperimeter calculator 328 determines the position of the perimeter oftumor 175 and/or certain points on the perimeter of tumor 175 such asanterior margin 214 and posterior margin 216. Location module 120 mayoutput items calculated by position calculator 324, direction calculator326, and/or tumor perimeter calculator 328 as 3D spatial properties 128.

Surgical cue generator 130 may include one or more of an incision sitecalculator 332, a projection calculator 334, a volumetric margincalculator 336, and a projection margin calculator 338. Incision sitecalculator 332 determines an optimal incision site for resection surgeryto remove tumor 175 based upon 3D spatial properties 128. In oneimplementation, incision site calculator 332 outputs point 222,determined by position calculator 324, as the incision site. Projectioncalculator 334 determines projection 224 based upon 3D spatialproperties 128, for example from line-of-sight 230 determined bydirection calculator 326, the perimeter of tumor 175 determined by tumorperimeter calculator 328, and additional positional informationspatially relating the perimeter of tumor 175 to surface 220 asdetermined by position calculator 324. Volumetric margin calculator 336determines the position of one or more margins of tumor 175 based upon3D spatial properties 128. In one implementation, volumetric margincalculator 336 determines distance 226 and/or distance 234. In thisimplementation, volumetric margin calculator 336 may determine distance226 from anterior margin 214 and point 222 determined by tumor perimetercalculator 328 and position calculator 324, respectively. Also in thisimplementation, volumetric margin calculator 336 may determine distance234 from posterior margin 216 and point 222 determined by tumorperimeter calculator 328 and position calculator 324, respectively. Inanother example, volumetric margin calculator outputs the position ofanterior margin 214 and/or the position of posterior margin 216, asdetermined by location module 120. Projection margin calculator 338determines the position of one or more margins of projection 224 such asthe positions of cranial margin 242, caudal margin 244, lateral margin246, and/or medial margin 248, for example based upon line-of-sight 230and an at least partial perimeter of tumor 175 determined by directioncalculator 326 and tumor perimeter calculator 328, respectively.Projection margin calculator 338 may utilize additional positionalinformation spatially relating the perimeter of tumor 175 to surface 220as determined by position calculator 324. Surgical cue generator 130outputs one or more items determined by incision site calculator 332,projection calculator 334, volumetric margin calculator, and/orprojection margin calculator 338 as surgical guidance cues 138.

In certain embodiments, system 100 includes a model generator 340 thatprocesses at least one supine image 158 to generate a model 348 ofbreast 172. In one example, model 348 includes a volumetric map of atleast a portion of breast 172 including tumor 175. In another example,model 348 includes a 3D surface map of at least a portion of surface174. In yet another embodiment, model 348 includes a volumetric map ofat least a portion of breast 172 including tumor 175 and at least aportion of surface 174. In a further embodiment, model 348 includesseveral of the above described maps. In one implementation, system 100further includes visualization module 140. In this implementation, modelgenerator 340 communicates model 348 to visualization module 140 suchthat visualization module 140 may superimpose one or more of surgicalguidance cues 138 on model 348. Visualization module 140 may displaymodel 348 with one or more of surgical guidance cues 138 superimposedthereon to surgeon 180 and/or communicate model 348 with one or more ofsurgical guidance cues 138 to an operating room (OR) navigation system,such as an OR navigation system known in the art. In one example, the ORnavigation system includes a tracking stylus, the position of which istracked in relation to model 348 and/or surgical guidance cues 138, suchthat the tracking stylus may mark one or more surgical guidance cues 138on breast 172. In another example, the OR navigation system includes anaugmented reality system that superimposes model 348 and/or surgicalguidance cues 138 on the view of surgeon 180. In yet another example,the OR navigation system includes a stereotactic device, the position ofwhich is tracked in relation to model 348 and/or surgical guidance cues138, such that the stereotactic device may mark one or more surgicalguidance cues 138 on or in breast 172.

In one usage scenario, each supine image 158 is captured by imagingmodule 150. In another usage scenario, imaging module 150 captures oneor more images 358 that need processing to determine supine images 158.For example, the position and shape of breast 172 may differ between (a)the positioning of patient 170 during preoperative imaging of breast 172upon which surgical guidance cues 138 are at least partly based and (b)the positioning of patient 170 during the resection surgery. In anembodiment, system 100 includes an image registration module 350 thatprocesses one or more images 358, captured when breast 172 is in aninitial preoperative position, and at least one additional supine image358, captured when breast 172 is in substantially the supine position inwhich the resection surgery is performed, to determine supine image(s)158. Supine images 158 and images 358 may be grayscale images or colorimages.

In one example, the initial preoperative position is the same as thesupine position in which the resection surgery is performed, and imageregistration module 350 may serve to register different types of imagescaptured by different imaging modalities, respectively. In this example,image registration module 350 may determine supine image 158 from avolumetric image 358 captured by volumetric image 152 or a third-partyvolumetric imager, wherein volumetric image 358 is captured while breast172 is substantially in the position associated with the resectionsurgery. Image registration module 350 may further determine a 3Dsurface image 358 captured by surface imager 154, or a third-partysurface image, wherein 3D surface image 358 is captured while breast 172is substantially in the position associated with the resection surgery.

In another example, the initial preoperative position is different fromthe supine position in which the resection surgery is performed, andimage registration module 350 further serves to correct for movement oftissue of breast 172. In this example, image registration module 350 maydetermine supine image 158 from (a) a volumetric image 358 captured byvolumetric image 152, or a third-party volumetric imager, while breast172 is in a position different from that associated with the resectionsurgery. Image registration module 350 may further determine a 3Dsurface image captured by surface imager 154, or a third-party surfaceimage, while the positioning of breast 172 is substantially the same asduring the resection surgery. In one implementation, images 358,captured when breast 172 is in a preoperative position different fromthe position in which the resection surgery is performed, are supineimages of breast 172. Even in this case, tissue displacement may existbetween the two supine positions. For example, tissue displacement maybe caused by a difference in hydration levels of patient 170, and/or theeffects of intraoperative surgical incisions compared with thepreoperative images. In another implementation, images 358, capturedwhen breast 172 is in a preoperative position different from theposition in which the resection surgery is performed, are prone imagesof breast 172.

Image registration module 350 includes a feature locator 356 thatidentifies features in images 358. Feature locator 356 facilitatesregistration of (a) an initial volumetric image 358 captured whilebreast 172 is in a position different from that associated with theresection surgery, to (b) a resection-associated 3D surface image 358captured while breast 172 is in the supine position associated with theresection surgery.

In one embodiment, image registration module 350 includes a rigid-bodytransformation unit 352 that applies a rigid-body transformation toinitial volumetric image 358 to register features located in bothinitial volumetric image 358 and resection-associated 3D surface image358, as determined by feature locator 356, to register initialvolumetric image 358 to resection-associated 3D surface image 358.Rigid-body transformation unit 352 may apply a translation, rotation,and/or a scaling to initial volumetric image 358. Rigid-bodytransformation unit 352 is applicable, for example, in scenarios whereinitial volumetric image 358 is a supine image. In another embodiment,image registration module 350 includes a deformable transformation unit354 that applies a deformable transformation to initial volumetric image358 to register features located in both initial volumetric image 358and resection-associated 3D surface image 358, as determined by featurelocator 356, to register initial volumetric image 358 toresection-associated 3D surface image 358. Herein, a “deformable”transformation (or registration) refers to a transformation (orregistration), respectively, that is potentially non-rigid. Thedeformable transformation allows accounting for deformation of breast172 in initial volumetric image 358 relative to resection-associated 3Dsurface image 358. Deformable transformation unit 354 is applicable inscenarios where initial volumetric image 358 is a prone image, forexample. Deformable transformation unit 354 may include a gravity unit355 that applies a transformation to initial volumetric image 358according to a gravitational model to account for gravitationallyinduced tissue displacement between initial volumetric image 358 andresection-associated 3D surface image 358. In one example, deformabletransformation unit 354 utilizes gravity unit 355 to transform a pronevolumetric image 358 according to a 2G gravitational force towards chestwall 240, to register a prone volumetric image 358 to a supineresection-associated 3D surface image 358. In yet another embodiment,image registration module 350 includes both rigid-body transformationunit 352 and deformable transformation unit 354. Without departing fromthe scope hereof, resection-associated 3D surface image 358 may begenerated from a plurality of captured resection-associated 3D surfaceimages 358. Likewise, initial volumetric image 358 may be generated froma plurality captured initial images 358.

Optionally, image registration module 350 includes a binary imagegenerator 357 that generates a binary version of image 358 to enhancecertain features, such as a surface 174.

Although not shown in FIG. 3, location module 120 may operate oncaptured images 358, and system 100 may implement a position correctionmodule to correct surgical guidance cues 138 instead of correctingcaptured supine images 358, without departing from the scope hereof.This position correction module would have functionality similar to thatof image registration module 350, however configured to operate on 3Dspatial properties 128 as opposed to images 358.

In certain embodiments, system 100 includes an interface 360 thatreceives user input such as the position of a user-defined incisionsite.

Without departing from the scope hereof, system 100 may be applied toimaging of other portions of patient 170 than breast 172, to guideresection of local tissue from patient 170. System 100 may be extendedto guide tumor resection from other body parts and organs, such as thebrain or the liver, as well as guide biopsy procedures of, e.g., muscleor bone, and also guide a core needle biopsy of a breast abnormalitydetected on MRI. As such, tumor 175 may be generalized to local tissueof patient 170, breast 172 may be generalized to a portion of patient170 associated with the resection surgery, surface 174 may begeneralized to a surface of patient 170 near the local tissue andincluding the incision site for removing the local tissue, and supineimage 158 may be generalized to an image of the portion of patient 170associated with the tissue resection procedure positioned as during thetissue resection procedure. System 100 may further be applied to guidelocal delivery of markers or to guide delivery of a therapeutic agent tolocal tissue of patient 170.

FIGS. 4A and 4B illustrate one exemplary method 400 for guidingresection of a tumor 175. Method 400, or a portion thereof, may beperformed by system 100. FIG. 4A shows a first portion of method 400 andFIG. 4B shows a second, optional portion of method 400. FIGS. 4A and 4Bare best viewed together.

In one embodiment, method 400 includes a step 410 of generating at leastone supine image 158 of breast 172. In one example, imaging module 150directly captures one or more supine images 158 or images 358 of breast172. In another example, system 100 receives images 358 from a thirdparty system. In certain embodiments, step 410 includes a step 412 ofregistering an initial volumetric image 358 of breast 172 to a 3Dsurface image 358 of surface 174, wherein the 3D surface image 358 issubstantially representative of the position of breast 172 associatedwith the resection surgery. In one example of step 412, imageregistration module 350 registers an initial volumetric image 358 to aresection-associated 3D surface image 358, as discussed above inreference to FIG. 3. Step 412 may include a step 414 of performing arigid-body or deformable transformation of initial volumetric image 358to register surface 174, as captured in initial volumetric image 358, tosurface 174 as captured in resection-associated 3D surface image 358.Optionally, step 414 utilizes fiducials on surface 174 identifiable ininitial volumetric image 358 and in resection-associated 3D surfaceimage 358. These fiducials may be fiducial markers placed on surface 174and/or natural features of surface 174, such as a nipple of breast 172.In one example of step 414, image registration module 350 utilizesrigid-body transformation unit 352 as discussed above in reference toFIG. 3. In another example of step 414, image registration module 350uses deformable transformation unit 354 as discussed above in referenceto FIG. 3.

In another embodiment, method 400 does not include step 410 but insteadreceives supine image(s) 158 generated by a third party system.

In a step 420, method 400 automatically determines surgical guidancecues 138. Step 420 includes steps 422 and 426. Step 422 processes atleast one supine image 158 to determine 3D spatial properties 128.Optionally, step 422 includes a step 424 of identifying tumor 175, andoptionally surface 174, in supine image(s) 158. In one example of step422, location module 120 determines 3D spatial properties 128, forexample as discussed above in reference to FIG. 3. Location module 120may utilize position calculator 324, direction calculator 326, and/ortumor perimeter calculator 328 to determine 3D spatial properties 128.Location module 120 may invoke feature locator 322 to identify tumor 175and/or at least a portion of surface 174 in supine image(s) 158. Step426 generates surgical guidance cues 138 based upon 3D spatialproperties 128. In one example of step 426, surgical cue generator 130generates surgical guidance cues 138 based upon 3D spatial properties128, for example as discussed above in reference to FIG. 3. Surgical cuegenerator 130 may utilize incision site calculator 322, projectioncalculator 334, volumetric margin calculator 336, and/or projectionmargin calculator 338 to generate surgical guidance cues 138.

Without departing from the scope hereof, step 422 may process capturedimages 358, and step 426 may include correcting surgical guidance cues138, determined based upon captured images 358, to account for tissuedisplacement of breast 172 between time of initial volumetric imagecapture and time of resection surgery.

In certain embodiments, method 400 includes a step 430 of visualizingsurgical guidance cues 138 relative to surface 174. In one suchembodiment, step 430 includes steps 432 and 433. Step 432 generates amodel 348 of surface 174, such as a 3D surface map of surface 174, andstep 433 superimposes one or more surgical guidance cues 138 on thismodel 348. In one example of steps 432 and 433, model generator 340generates a model 348 of surface 174 and visualization module 140superimposes one or more surgical guidance cues 138 on this model 348.In another such embodiment, step 430 includes steps 434 and 435. Step434 generates a volumetric model 348 of breast 172 including a model ofat least a portion of surface 174, such as a volumetric map of breast172 including a map of at least a portion of surface 174. Step 435superimposes one or more surgical guidance cues 138 on this volumetricmodel 348. Optionally, volumetric model 348 includes at least a portionof chest wall 240. In one example of steps 434 and 435, model generator340 generates a volumetric model 348 of surface 174 and visualizationmodule 140 superimposes one or more surgical guidance cues 138 on thisvolumetric model 348.

In an optional step 436, method 400 communicates model 348 with one ormore surgical guidance cues 138 superimposed thereon, as generated instep 430, to a surgical navigation system such as an OR navigationsystem discussed in reference to FIG. 1.

Another optional step 438 produces a patient-specific locator form thatfits surface 174 and includes features that indicate one or more ofsurgical guidance cues 138 or enable transfer of one or more surgicalguidance cues 138 onto breast 172. Such a patient-specific locator formis discussed further in reference to FIGS. 20-27.

Method 400 may include a step 440 of transferring one or more surgicalguidance cues 138 to surface 174. In one example of step 440, thenavigation system of step 436 is used to transfer one or more surgicalguidance cues 138 to surface 174. In one example, surgeon 180 uses atracking stylus to transfer one or more surgical guidance cues 138 tosurface 174 and/or uses a tracked stereotactic device to transfer one ormore surgical guidance cues 138 to interior locations of breast 172 and,optionally, to surface 174. In another example of step 440, surgeon 180places the locator form of step 438 on breast 172 and uses features ofthe locator form to transfer one or more surgical guidance cues 138 tosurface 174 and/or interior locations of breast 172.

Without departing from the scope hereof, method 400 may be extended to amore general embodiment of guiding resection of local tissue frompatient 170. Method 400 may be extended to guide tumor resection fromother body parts and organs, such as the brain or the liver, as well asguide biopsy procedures of, e.g., breast 172, muscle, or bone. As such,tumor 175 may be generalized to local tissue of patient 170, breast 172may be generalized to a portion of patient 170 associated with theresection surgery, surface 174 may be generalized to a surface ofpatient 170 near the local tissue and including the incision site forremoving the local tissue, and supine image 158 may be generalized to animage of the portion of patient 170 associated with the tissue resectionprocedure positioned as during the tissue resection procedure. Method400 may further be applied to guide local delivery of markers or atherapeutic agent to patient 170.

FIG. 5A illustrates one exemplary method 500 for generating surgicalguidance cues 138 from at least one supine image 158. Method 500 isperformed by location module 120 and surgical cue generator 130, forexample. Method 500 is an embodiment of step 420 of method 400.

In a step 510, method 500 calculates the position P1 of the centroid oftumor 175. In one example of step 510, position calculator 324calculates the position of centroid 212.

In a step 520, method 500 calculates the point P2 on surface 174, whichis closest to tumor 175. In one example of step 510, position calculator324 calculates the position of point 222.

In a step 525, method 500 determines the optimal incision site as pointP2. In one example of step 525, surgical cue generator 130 determinesthe optimal incision site as point 222.

In a step 530, method 530 determines a vector V1 from centroid P1 topoint P2. In one example of step 530, direction calculator 326determines a vector from centroid 212 to point 222, wherein this vectoris located along line-of-sight 230.

In a step 532, method 500 projects a silhouette of tumor 175 ontosurface 174 along vector V1 to determine the projection Si of tumor 175onto surface 174. In one example of step 532, projection calculator 334projects a silhouette of tumor 210 onto surface 220 along line-of-sight230 to form projection 224. Optionally, method 500 includes a step 534of determining one or more margins of the projection Si. In one exampleof step 534, projection margin calculator 338 determines the position ofcranial margin 242, caudal margin 244, lateral margin 246, and/or medialmargin 248 of projection 224.

In a step 536, method 500 determines the anterior margin of tumor 175 asthe distance between (a) the intersection of point P2 and vector V1 and(b) the perimeter of tumor 175. In one example of step 536, volumetricmargin calculator 336 determines the position of anterior margin 214and, subsequently, determines distance 226.

In an optional step 540, method 500 extends vector V1 to the chest wallof patient 170 to define a line L1. In one example of step 540,direction calculator 326 extends line-of-sight 230 from centroid 212 tochest wall 240. Method 500 may include a step 542 of determining theposterior margin of tumor 175 as the distance between (a) theintersection of line L1 and the posterior perimeter of tumor 175 and (b)the intersection of line L1 and the chest wall of patient 170. In oneexample of step 542, volumetric margin calculator 336 determines theposition of posterior margin 216 and, subsequently, determines distance234.

FIGS. 5B and 5C illustrates one exemplary method 550 that is analternative embodiment of method 500. FIG. 5B is a flowchart for method550, and FIG. 5C is a diagram showing certain parameters of method 550.FIGS. 5B and 5C are best viewed together.

In a step 560, method 550 fits an ellipsoid to tumor 175. In one exampleof step 560, position calculator 324 fits an ellipsoid 502 to tumor 210.

A step 570 determines the direction of the major axis of the ellipsoidof step 560 to define line L1 as the line having same direction as themajor axis. The major axis of the ellipsoid is the axis associated withthe longest semi-axis of the ellipsoid. In one example of step 570,direction calculator 326 determines the direction of the major axis ofellipsoid 502 to define line 504.

A step 572 determines the optimal incision site as the point located atthe intersection of line L1 and surface 174. In one example of step 572,position calculator 324 determines point 222 located at the intersectionof line 504 with surface 220, and incision site calculator 332 definesthe optimal incision point as point 222.

A step 574 determines the projection of tumor 175 onto surface 174 asthe projection of the silhouette of tumor 175 onto surface 174 alongline L1. In one example of step 574, tumor perimeter calculator 328determines the perimeter of tumor 210, and projection calculator 334project this perimeter onto surface 220 along line 504. Steps 578, 580and 582 are similar to steps 536, 540, and 542, respectively, except forbeing based upon line L1 co-directional with the major axis of ellipsoid502 (as opposed to line L1 defined by vector V1).

FIG. 6 illustrates one exemplary method 600 for generating surgicalguidance cues 138 from at least one supine image 158 and a user-definedincision site for resection of tumor 175. Method 600 is performed bylocation module 120 and surgical cue generator 130, for example. Method600 enables system 100 to generate surgical guidance cues 138 whiletaking into account a user-defined incision site. Method 600 is anembodiment of step 420 of method 400.

Method 600 includes step 510 of calculating centroid P1 of tumor 175.Method 600 also includes a step 610 of receiving, as point P2, theposition of a user-defined incision site for performing the resectionsurgery. In one example of step 610, interface 360 receives the positionof a user-defined incision site. Next, method 600 proceeds to performsteps 530, 532, 536, and optionally one or more of steps 534, 540, and542, as discussed in reference to FIG. 5A except with P2 being theposition of the user-defined incision site.

Without departing from the scope hereof, each of methods 500, 550, and600 may be extended to a more general embodiment of guiding resection oflocal tissue from patient 170. Each of methods 500, 550, and 560 may,for example, be extended to guide tumor resection from other body partsand organs, such as the brain or the liver, as well as guide biopsyprocedures of, e.g., breast 172, muscle, or bone, or extended to guidelocal delivery of markers or a therapeutic agent to patient 170. Assuch, each of tumors 175 and 210 may be generalized to local tissue ofpatient 170, each of breasts 172 and 202 may be generalized to a portionof patient 170 associated with the tissue resection procedure, each ofsurfaces 174 and 220 may be generalized to be a surface of patient 170near the local tissue and including the incision site for removing thelocal tissue, and chest wall 240 may be generalized to be another tissuetype or organ underlying the local tissue when viewed from the surface.

FIG. 7 illustrates one exemplary method 700 for generating supine images158, wherein supine images 158 include separate volumetric and 3Dsurface images. Method 700 is an embodiment of step 410 of method 400.Method 700 is performed by imaging module 150 and image registrationmodule 350, for example.

In a step 710, method 700 captures at least one volumetric image 358 ofbreast 172, including tumor 175 and with breast 172 in the supineposition associated with the resection surgery. Step 710 is performed byvolumetric imager 152, for example.

In a step 720, method 700 captures at least one 3D surface image 358 ofbreast 172, including tumor 175 and with breast 172 in the supineposition associated with the resection surgery. Step 720 is performed bysurface imager 154, for example.

In one embodiment, method 700 further includes a step 730 of performinga rigid-body transformation of (a) the volumetric image 358 captured instep 710 to register surface 174 as captured in the volumetric image,with (b) surface 174 as captured in the 3D surface image 358 of step720, so as to register volumetric image 358 with 3D surface image 358.Step 730 is performed by image registration module 350, for example.

In another embodiment, volumetric image 358 of step 710 and 3D surfaceimage 358 of step 720 are inherently registered and method 700 does notinclude step 730. Instead, method 700 outputs volumetric image 358 ofstep 710 and 3D surface image 358 of step 720 as supine images 158. Inone example, volumetric imager 152 and surface imager 154 capturevolumetric image 358 and 3D surface image 358, respectively, in absolutecoordinates, and thus ensure inherent registration of the volumetricimage and the 3D surface image.

Without departing from the scope hereof, method 700 may replace steps710 and 720 by a step of receiving volumetric image 358 of step 710 and3D surface image 358 of step 720 from one or more third party imagingsystems.

FIG. 8 illustrates one exemplary method 800 for generating at least onesupine image 158, wherein each of the at least one supine image 158 is avolumetric image. Method 800 is performed by volumetric imager 152, forexample. Method 800 is an embodiment of step 410.

Method 800 includes a step 810 of capturing at least one volumetricsupine image 158 of breast 172 in the supine position associated withthe resection surgery to remove tumor 175. The at least one volumetricsupine image 158 includes an image of tumor 175 and an image of at leasta portion of surface 174. In one example of step 810, a volumetricsupine image 158 includes both an image of tumor 175 and an image of atleast a portion of surface 174. In another example, step 810 captures aplurality of volumetric supine images 158, wherein at least onevolumetric supine image 158 provides an image of tumor 175 and at leastone other volumetric supine image 158 provides an image of at least aportion of surface 174.

FIG. 9 illustrates one exemplary method 900 of generating at least onesupine image 158 while accounting for tissue displacement of breast 172between at least a portion of preoperative imaging and resectionsurgery. Method 900 is performed by imaging module 150 and imageregistration module 350, for example. Method 900 is an embodiment ofstep 410.

In a step 910, method 900 captures at least one initial volumetric image358 of breast 172 in an initial supine position. This at least oneinitial volumetric image 358 includes an image of tumor 175 and an imageof at least a portion of surface 174, as discussed in reference to FIG.8. In one example of step 910, volumetric imager 152 captures at leastone initial volumetric image 358 of breast 172.

In a step 920, method 900 captures at least one 3D surface image 358 ofbreast 172 in the supine position associated with resection of tumor175. In one example of step 920, surface imager 154 captures a 3Dsurface image 358 of at least a portion of surface 174. In anotherexample of step 920, volumetric imager 152 captures a volumetric image358 of breast 172, which includes at least a portion of surface 174.

In a step 930, method 900 registers initial volumetric image 358,captured in step 910 to 3D surface image 358 captured in step 920. Step930 is an embodiment of step 412 and may be performed by imageregistration module 350. Step 930 may include a step 932 of performing arigid-body transformation of initial volumetric image 358 to registersurface 174, as captured in initial volumetric image 358, to surface 174as captured in resection-associated 3D surface image 358. Step 932 is anembodiment of step 412 and may utilize fiducials on surface 174 asdiscussed in reference to FIGS. 4A and 4B. In one example of step 932,image registration module 350 utilizes rigid-body transformation unit352 as discussed above in reference to FIG. 3.

Without departing from the scope hereof, method 900 may replace steps910 and 920 by a step of receiving initial volumetric image 358 of step910 and 3D surface image 358 of step 920 from one or more third partyimaging systems.

Without departing from the scope hereof, each of methods 700, 800, and900 may be extended to a more general embodiment of guiding resection oflocal tissue from patient 170. Each of methods 700, 800, and 900 may beextended to guide tumor resection from other body parts and organs, suchas the brain or the liver, as well as guide biopsy procedures of, e.g.,breast 172, muscle, or bone, extended to guide local delivery of markersor a therapeutic agent to patient 170. As such, tumor 175 may begeneralized to local tissue of patient 170, breast 172 may begeneralized to a portion of patient 170 associated with the tissueresection procedure, surface 174 may be generalized to a surface ofpatient 170 near the local tissue and including the incision site forremoving the local tissue, and supine image 158 may be generalized to animage of the portion of patient 170 associated with the tissue resectionprocedure positioned as during the tissue resection procedure. Each ofmethods 700, 800, and 900 may further be used to guide local delivery ofmarkers or a therapeutic agent to patient 170

FIGS. 10A and 10B illustrate one exemplary method 1000 for guidingresection of local tissue from patient 170. Method 1000, or a portionthereof, may be performed by system 100. FIG. 10A shows a first portionof method 1000 and FIG. 10B shows a second optional portion of method1000. FIGS. 10A and 10B are best viewed together. Method 1000 is anextension of method 400 to general resection of local tissue frompatient 170.

In one embodiment, method 1000 includes a step 1010 of generating atleast one image 158 of a portion of patient 170 that is associated withthe tissue resection procedure. This resection-associated portion ofpatient 170 includes (a) the local tissue to be resected and (b) asurface of patient 170 near the local tissue and including the incisionsite for the tissue resection procedure. Step 1010 is a generalizationof step 410. Image 158 generated by step 1010 is representative of theresection-associated portion of patient 170 substantially as positionedduring the tissue resection procedure. In this more general embodiment,image 158 need not be a supine image. In one example of step 1010,imaging module 150 directly captures one or more images 158 or images358 of the resection-associated portion of patient 170. In anotherexample, system 100 receives images 358 from a third party system. Incertain embodiments, step 1010 includes a step 1012 of registering aninitial volumetric image 358 of the local tissue to a 3D surface image358 of the associated surface of patient 170, wherein 3D surface image358 is substantially representative of the resection-associated portionof patient 170 as positioned during the tissue resection procedure. Inone example of step 1012, image registration module 350 registers aninitial volumetric image 358 to 3D surface image 358, as discussed abovein reference to FIG. 3. Step 1012 may include step 414 as discussedabove in reference to FIGS. 4A and 4B but extended to general resectionof local tissue from patient 170.

In another embodiment, method 1000 does not include step 1010 butinstead receives image(s) 158 as generated by a third party system.

In a step 1020, method 1000 automatically determines surgical guidancecues 138 indicating 3D spatial properties of the local tissue to beresected. Step 1020 includes steps 1022 and 1026, and optionally a step1024. Steps 1020, 1022, 1024, and 1026 are extensions of respectivesteps 420, 424, 424, and 426 to general resection of local tissue frompatient 170.

Step 1022 processes at least one image 158 to determine 3D spatialproperties 128 for the local tissue. Step 1022 may include step 1024.Step 1024 identifies the local tissue to be resected and, optionally,identifies the associated surface of patient 170. Step 1026 utilizes 3Dspatial properties 128 to generate surgical guidance cues 138 for thelocal tissue with respect to the associated surface of patient 170.Steps 1020, 1022, 1024, and 1026 are performed in a manner similar tothat discussed above in reference to FIGS. 4A and 4B for steps 420, 424,424, and 426.

In certain embodiments, method 1000 includes a step 1030 of visualizingsurgical guidance cues 138 relative to the associated surface of patient170. Step 1030 is an extension of step 430 to general resection oftissue from patient 170. In one implementation, step 1030 includes steps1032 and 1033. Step 1032 generates a model 348 of the surface of patient170 associated with the tissue resection procedure, such as a 3D surfacemap of this surface, and step 1033 superimposes one or more surgicalguidance cues 138 on this model 348. Steps 1032 and 1033 are performedin a manner similar to that of steps 432 and 433. In anotherimplementation, step 1030 includes steps 1034 and 1035. Step 1034generates a volumetric model 348 of the resection-associated portion ofpatient 170 including a model of at least a portion of theresection-associated surface, such as a volumetric map of theresection-associated portion of patient 170 including a map of at leasta portion of the resection-associated surface, and step 1035superimposes one or more surgical guidance cues 138 on this volumetricmodel 348. Steps 1032 and 1033 are performed in a manner similar to thatof steps 432 and 433.

Optionally, method 1000 includes step 436 as discussed above inreference to FIGS. 4A and 4B.

Method 1000 may include a step 1038 of producing a patient-specificlocator form that fits the resection-associated surface of patient 170and includes features that indicate one or more of surgical guidancecues 138 and/or enable transfer of surgical guidance cues 138 to patient170. Such a patient-specific locator form is discussed further inreference to FIGS. 20-27. Step 1038 is an extension of step 438 togeneral resection of local tissue from patient 170.

Method 1000 may further include a step 1040 of transferring one or moresurgical guidance cues 138 to the resection-associated surface ofpatient 170. Step 1040 is performed in a manner similar to that of step440.

FIG. 11 illustrates one exemplary method 1100 for generating at leastone image 158 of a portion of patient 170 including local tissue to beresected, while accounting for tissue displacement between at least aportion of pre-operative imaging and the resection procedure. Method1100 is performed by imaging module 150 and image registration module350, for example. Method 1100 is an embodiment of step 1010 and is anextension of method 900 to general resection of local tissue frompatient 170.

In a step 1110, method 1000 captures at least one initial volumetricimage 358 of the local tissue in an initial position. This at least oneinitial volumetric image 358 includes an image of the local tissue to beresected and an image of at least a portion of a surface of patient 170associated with the resection procedure, as discussed in reference toFIGS. 10A and 10B. In one example of step 1110, volumetric imager 152captures at least one initial volumetric image 358 of a portion ofpatient 170 associated with the resection procedure.

In a step 1120, method 1100 captures at least one 3D surface image 358of the resection-associated surface of patient 170 as positioned duringthe resection procedure. Step 1120 may be performed in a manner similarto that of step 920.

In a step 1130, method 1100 registers the initial volumetric image 358,captured in step 1110, to the 3D surface image 358 captured in step1120. Step 1130 is an extension of step 930 and may be performed byimage registration module 350. Step 1130 may include step 932.

FIG. 12 illustrates one exemplary method 1200 for using a rigid-bodytransformation to register (a) an initial volumetric image 358 of aportion of patient 170 associated with resection surgery to (b) a 3Dsurface image 358 of the portion of patient 170 associated withresection surgery substantially as positioned during resection surgery.Method 1200 is an embodiment of step 932 and of step 414, and may beimplemented by rigid-body transformation unit 352 and feature locator356 in cooperation.

In a step 1210, method 1200 identifies the spatial locations offiducials on patient 170 in initial volumetric image 358 and in 3Dsurface image 358. The fiducials may be fiducial markers placed on thesurface of patient 170 and/or natural features of patient 170. In oneexample of step 1210, feature locator 356 identifies fiducials onsurface 174 in initial volumetric image 358 and identifies the samefiducials in 3D surface image 358. Depending on the tissue displacementbetween initial volumetric image 358 and in 3D surface image 358, thefiducials may be in relatively similar positions, for example as betweena supine initial volumetric image 358 of breast 172 and a supine 3Dsurface image 358 of breast 172, or be relatively disparate, as betweena prone initial volumetric image 358 of breast 172 and a supine 3Dsurface image 358 of breast 172.

In a step 1220, method 1200 performs a rigid-body transformation ofinitial volumetric image 358 to register fiducials identified in initialvolumetric image 358 with the corresponding fiducials identified in 3Dsurface image 358, by matching the fiducials identified in thecoordinate system of initial volumetric image 358 with the correspondingfiducials identified in the coordinate system of 3D surface image 358.Step 1220 thus produces a registered volumetric image that is registeredto 3D surface image 358. This registered volumetric image is an exampleof supine image 158. This rigid-body transformation may includetranslation, rotation, and/or scaling, but generally does not requireshearing. In one example of step 1220, rigid-body transformation unit352, according to fiducial locations identified by feature locator 356in step 1210, applies a rigid-body transformation to initial volumetricimage 358 to register fiducials identified in initial volumetric image358 with the corresponding fiducials identified in 3D surface image 358.

FIG. 13 illustrates one exemplary method 1300 for using a rigid-bodytransformation to register an initial volumetric image 358 of patient170 to a 3D surface image 358 of patient 170 reflecting the positioningused during the resection procedure, and further using binary imagegeneration and deformable transformation thereof to emphasize certainfeatures. Method 1300 is an embodiment of step 414, and may beimplemented by deformable transformation unit 354 and feature locator356 in cooperation. Method 1300 may be particularly useful when thetissue displacement between initial volumetric image 358 and 3D surfaceimage 358 is significant, for example as generally is the case for aprone volumetric image 358 of breast 172 and a corresponding supine 3Dsurface image 358 of surface 174. Generally, method 1300 may be usefulin scenarios where there is tissue displacement between initialvolumetric image 358 of a portion of patient 170 and a corresponding 3Dsurface image 358 of a surface of patient 170, or in scenarios where adifference between the imaging modalities used to capture initialvolumetric image 358 and 3D surface image 358 results in an apparenttissue displacement.

Method 1300 includes steps 1210 and 1220 as discussed above in referenceto FIG. 12.

In a subsequent step 1330, method 1300 creates a binary image version ofthe registered volumetric image generated in step 1320. In one exampleof step 1330, binary image generator 357 processes the registeredvolumetric image to create a binary version thereof. Step 1330 may beimplemented by binary image generator 357.

In a step 1340, method 1300 creates a binary image version of 3D surfaceimage 358. In one example of step 1340, binary image generator 357processes 3D surface image 358 to create a binary version thereof. Step1340 may be implemented by binary image generator 357.

Both 3D surface image 358 and initial volumetric image 358 may begrayscale images, where each pixel (for 3D surface image 358) or voxel(for initial volumetric image 358) is represented as a gray color in arange between darkest (black) or lightest (white). Alternatively, one orboth of 3D surface image 358 and initial volumetric image 358 is a colorimage that also provides intensity information. In contrast, a binaryimage represents each pixel or voxel as either black or white, withoutany shades of gray in between. Therefore, creating a binary image from agrayscale image is a technique for emphasizing certain desired featuresin an image, and removing other features from the image.

In one example, step 1330 generates a binary version of the registeredvolumetric image, which emphasizes surface features in the registeredvolumetric image, such as features of surface 174 of breast 172. Forexample, step 1330 may create the binary version of the registeredvolumetric image from the registered volumetric image by setting theintensity values of the voxels that are sufficiently close to thesurface to unity and zero otherwise. A typical range for voxels definedas being sufficiently close to the surface may be, but is not limitedto, voxels within 10 mm of the tissue-to-air interface.

Steps 1330 and 1340 may serve to emphasize features common to both theregistered volumetric image and 3D surface image 358 in order to performa deformable transformation of the binary version of the registeredvolumetric image to the binary version of 3D surface image 358. Both ofthese binary images highlight the same tissue-to-air interface, whichallows for non-rigid registration of the two binary images to bedeformably registered using the rigid registration of step 1320 as astarting point.

In a step 1350, method 1300 deformably transforms the binary version ofthe registered volumetric image to register the binary version of theregistered volumetric image to the binary version of 3D surface image358. Step 1350 may be performed by deformable transformation unit 354.

In one example, step 1350 first performs an affine registration of thebinary version of the registered volumetric image to the binary versionof 3D surface image 358 to produce an intermediate binary image. Next,in this example, step 1350 applies a deformable transformation to theintermediate binary image to register the intermediate binary image tothe binary version of 3D surface image 358. This deformabletransformation is subject to the constraint of matching fiducials in theintermediate binary image to those in the binary version of 3D surfaceimage 358. The deformable transformation may be, but is not limited to,a B-Spline deformable registration, or another deformable transformationor registration known in the art.

FIG. 14 illustrates one exemplary method 1400 for generating a binaryversion of the registered volumetric image of step 1320 of method 1300.Method 1400 is an embodiment of step 1330. Method 1400 is performed bybinary image generator 357, for example.

A step 1410 determines a gradient image version of the registeredvolumetric image of step 1220 as implemented in method 1300. Thisgradient image indicates directional change in the intensity or color ofthe registered volumetric image and may be used to identify certainfeatures in the registered volumetric image. The gradient image mayidentify one or more anatomical boundaries, for example a tissue-to-airinterface such as surface 174.

A step 1420 dilates voxels at the tissue-to-air interface. Step 1420 mayutilize a similar range of voxels as discussed above for step 1330 inreference to FIG. 13.

A step 1430 filters out voxels, of the image generated in step 1420,that are below a threshold intensity. While in general, the thresholdintensity level may be chosen to be any intensity, for example toprovide a more granular grayscale image, for a binary image thethreshold may be set to maximum intensity, so that any pixels or voxelsthat were not set to maximum intensity during dilation are filtered out.

Additional pre-registration processing, such as rasterization, Gaussiansmoothing, and morphology operations, is possible in order to furtherimprove the robustness of image registration such as the rigid-bodyregistration performed in step 1220. In one example, the dilatedvolumetric gradient image generated in step 1420 as well as 3D surfaceimage 358 may be Gaussian-smoothed (for example, with a kernel of 5×5×5voxels) to reduce the noise level. Registration of the dilatedvolumetric gradient image to 3D surface image 358 may be performed. Forexample, the dilated volumetric gradient image may be registered to arasterized version of 3D surface image 358. Registration may be based onmaximization of mutual information between the two image volumes.

Without departing from the scope hereof, each of methods 1200, 1300, and1400 may utilize additional processing or filtering of images 358 tooptimize registration for specific types of images 358. The goal of suchparameter optimization may be to emphasize features in initialvolumetric image 358 to match similar features that are inherentlyemphasized in a corresponding 3D surface image 358 representative of theresection-associated portion of patient 170 being in substantially theposition used during the resection procedure. These parametermanipulations may be based on a predetermined set of parametersdepending upon the type of 3D surface image 358 or, alternatively, maybe optimized based on conditions particular to a 3D surface image 358.

FIG. 15 illustrates one exemplary finite element modeling (FEM) method1500 for deformably registering a prone volumetric image 358 of breast172 to a resection-associated supine 3D surface image 358 of surface174. FEM method 1500 is an embodiment of step 414, and may becooperatively implemented by feature locator 356 and an embodiment ofdeformable transformation unit 354 that includes gravity unit 355.Generally, FEM method 1500 may be useful in scenarios where tissuedisplacement between prone volumetric image 358 and supine 3D surfaceimage 358 is such that a rigid-body transformation is inadequate. FIGS.16A-C and 17A-C discussed below show exemplary tissue displacements forbreast 172 between prone and supine positions.

FEM method 1500 takes into account the physical properties of the tissuein the images. By modeling the physical properties of the tissue, FEMmethod 1500 may more accurately register prone volumetric image 358 tosupine 3D surface image 358.

FEM method 1500 leverages information about the tissue in the volumetricand surface images to more accurately model the transformation. Forexample, tissue of breast 172 may exhibit different elastic propertiesin a first region of breast 172 than the elastic properties of a secondregion of breast 172. For example, regions of breast 172 correspondingto gland tissue may be assigned a first elastic modulus, and regions ofbreast 172 where invasive ductal carcinoma is detected may be assigned asecond elastic modulus, where the second elastic modulus is greater thanthe first elastic modulus. Examples of other tissues with known elasticmodulus, include, but are not limited to, normal fat tissue, normalgland tissue, fibrous tissue, invasive ductal carcinoma, and ductalcarcinoma in situ (DCIS). Whereas a rigid transformation may produceerrors by not taking the different elastic properties of differentregions of tissue into account, FEM method 1500 may produce moreaccurate results based on more accurate modeling of the properties ofdifferent regions of tissue.

In a step 1510, FEM method 1500 assigns material properties toidentified tissue in the volumetric scan. For example, chest wall 240may be treated as being relatively inelastic, and may therefore onlydeform minimally between prone and the supine positions. In contrast,gland and fat tissue may have higher elasticity properties, while muscleor some types of cancerous tissue may have lower elasticity properties.By identifying types of tissue and assigning different materialproperties and properties to the different types of tissue based uponthe identified tissue type, FEM method 1500 may more accurately modelthe deformation between the prone and supine positions.

In a step 1520, FEM method 1500 deforms prone volumetric image 358 witha finite-element model by applying a computational model to pronevolumetric image 358. This computational model applies a simulatedgravitational force of 2G to breast 172 in the direction of chest wall240. This deformation attempts to normalize prone volumetric image 358with supine 3D surface image 358, since supine 3D surface image 358 isperformed while breast 172 is subject to a gravitational force of 1G inthe direction toward chest wall 240 and prone volumetric image 358 istaken when breast 172 is subject to a gravitational force of 1G in thedirection away from chest wall 240. Step 1520 may be implemented bygravity unit 355.

In a step 1530, FEM method 1500 performs a rigid-body transformation ofthe deformed volumetric image generated in step 1520 to register thedeformed volumetric image to supine 3D surface image 358. Step 1530 thusgenerates a registered volumetric image that is registered to supine 3Dsurface image 358. This registered volumetric image is an example ofsupine image 158.

An optional step 1540 generates displacement vectors, for examplebetween surface 174 as shown in the registered volumetric image andsurface as shown in 3D surface image 358.

An optional step 1550 refines the registered volumetric image generatedin step 1530. In one embodiment, step 1550 performs a second deformationsimulation. In another embodiment, the displacement vectors of step 1540are generated by matching the set of fiducial locations and step 1550deformably transforms the shape of breast 172 in an inverse modelingapproach to avoid overfitting the shape that can be adversely affectedby measurement error.

In an alternative embodiment, FEM method 1500 is used to register asupine volumetric image 358 to a supine 3D surface image 358. Inaccordance with this alternative embodiment, the flowchart of FIG. 15would be slightly modified, with step 1520 reading “apply a 2Ggravitational force adjustment toward chest wall”.

Without departing from the scope hereof, FEM method 1500 may be extendedfrom registration of images of breast 172 to registration of images ofother portions of patient 170 associated with a different tissueresection procedure by using different mathematical models for thetissue in the images. For example, while a simple linear elastic modelmay be used to model tissue of breast 172, additional models used by amore generally applicable embodiment of FEM method 1500 may include, butare not limited to, linear elastic, neohookean, exponential, and othernon-linear approaches.

In each of methods 1200, 1300, 1400, and 1500, surface 174 (or anothersurface of patient 170 associated with a tissue resection procedure) maybe modeled by mapping the surface as a mesh of elements. The elementsmay be represented by polygons, for example, triangles. Deformationcalculations, as used in FEM method 1500, may be simplified bycalculating displacement vectors for specific points, or nodes, on themapping mesh, rather than calculating displacement vectors for everypoint on the surface. The nodes may be, for example, the comers of eachtriangle in the mapping mesh. Once the location of the nodes iscalculated after deformation, the intermediate points may beapproximated, for example, by linear interpolation. Similarly, mappingmeshes may be used to correlate surface locations in the volumetricimage to locations on the 3D surface image. One exemplary approach is tomap points on the surface of the volumetric image to the closest nodelocations (after rigid-body transformation) on the 3D surface image. The3D surface image mesh may typically have much higher density, sointerpolation between nodes is not needed. An alternative approach is tofind the closest point on each element of the 3D surface image, asopposed to the closest node.

FIGS. 16A-C and 17A-C illustrate exemplary tissue displacement forbreast 172 between prone and supine positions for three patients 170.FIGS. 16A, 16B, and 16C show breasts 172(1), 172(2), and 172(3) of threerespective patients 170 in prone position. The perspective in FIGS.16A-C is from above the head of each patient 170. FIGS. 17A, 17B, and17C show breasts 172(1), 172(2), and 172(3) when patients 170 are insupine position. The perspective in FIGS. 17A-C is from above eachpatient 170. Each of FIGS. 16A-C and 17A-C indicate chest wall outline240, surface 220, tumor 175 (or alternatively other local tissue ofinterest), a nipple 1610, and an imaginary center axis 1620 is shown,indicating a midpoint of the breasts 172(1), 172(2), and 172(3) relativeto chest wall 240. FIGS. 16A-C and 17A-C are best viewed together.

Breast 172(1) in supine position demonstrates both vertical compressionand horizontal displacement, with both nipple 1610 and tumor 175distorted in relation to center axis 1620 in comparison breast 172(1) inprone position. Breast 172(2) shows both vertical compression andhorizontal displacement from prone position to supine position, withboth nipple 1610 and the tumor 175 having been distorted in relation tothe center axis 1620. However, the relational positions of the elementsof breast 172(2) are different from those of breast 172(1). Suchvariations may be due to the amount of breast tissue, the relativedensity of the breast tissue, and/or the amount of surface area. Incontrast, while breast 172(3) demonstrates some vertical compression,breast 172(3) demonstrates significantly less horizontal displacement.

FIGS. 18A-D show one example of rigid-body transformation of avolumetric image 1810 of breast 172, captured with breast 172 in aninitial supine position, to register volumetric image 1810 to a 3Dsurface image 1820 captured with breast 172 in the supine position usedduring resection of tumor 175. The rigid-body transformation shown inFIGS. 18A-D is performed according to method 1200. Volumetric image 1810is an example of volumetric image 358. 3D surface image 1820 is anexample of 3D surface image 358. FIG. 18A shows surface 174 as capturedin volumetric image 1810. FIG. 18B shows 3D surface image 1820. FIG. 18Cshows surface 174 of volumetric image 1810 after registration ofvolumetric image 1810 to optical image 1820. FIG. 18D shows across-section of volumetric image 1810, including tumor 175, afterregistration of volumetric image 1810 to optical image 1820. FIGS. 18A-Dare best viewed together.

As illustrated in FIGS. 18A and 18B, step 1210 of method 1200 identifies(a) fiducial marker locations 1812 on surface 174 in volumetric image1810 and (b) fiducial marker locations 1822 on surface 174 in 3D surfaceimage 1820. As illustrated in FIGS. 18C and 18D, step 1220 of method1200 performs a rigid-body transformation of volumetric image 1810 tomatch fiducial marker locations 1812 to fiducial marker locations 1822.This results in registration of surface 174, as captured in volumetricimage 1810, to surface 174 as captured in 3D surface image 1820. FIG.18D shows tumor 175 and the location of tumor 175 in relation to surface174.

FIG. 19 illustrates one exemplary computer 1900 that implements locationmodule 120, surgical cue generator 130, and optionally imageregistration module 350. Computer 1900 may be implemented in system 100.Computer 1900 may perform at least a portion of any of methods 400, 500,550, 600, 900, 1000, 1100, 1200, 1300, 1400, and 1500.

Computer 1900 includes a non-transitory memory 1910, a processor 1980,and an interface 1990. Processor 1980 is communicatively coupled withmemory 1920 and interface 1990. Memory 1910 is, for example, of typeROM, Flash, magnetic tape, magnetic drive, optical drive, RAM, othernon-transitory medium, or combinations thereof. Interface 1990implements (a) an interface between location module 120, or imageregistration module 350, and imaging module 150 and (b) an interfacebetween surgical cue generator 360 and visualization module 140.Interface 1990 may further implement interface 360. Interface 1990 isfor example a wired interface (such as Ethernet, USB, FireWire, orThunderbolt), a wireless interface (such as IEEE 802.11, Wi-Fi, orBluetooth), and/or a user interface such as a display, touchscreen,keyboard, and/or pointing device. Memory 1910 includes software 1920encoded in memory 1910 as machine-readable instructions executable byprocessor 1980. Memory 1910 further includes a data storage 1970.Software 1920 includes location instructions 1930, surgical cuegeneration instructions 1940, and optionally image registrationinstructions 1950.

Processor 1980 may execute (a) location instructions 1930 to implementlocation module 120 and (b) surgical cue generation instructions 1940 toimplement surgical cue generator 130. In embodiments of computer 1900that include image registration instructions 1950, processor 1980 mayexecute image registration instructions 1950 to implement imageregistration module 350.

Location instructions 1930 may include one or more of feature locationinstructions 1932, position calculation instructions 1934, directioncalculation instructions 1936, and tumor perimeter location instructions1938. Processor 1980 may execute feature location instructions 1932,position calculation instructions 1934, direction calculationinstructions 1936, and/or tumor perimeter location instructions 1938 toimplement feature locator 322, position calculator 324, directioncalculator 326, and/or tumor perimeter calculator 328, respectively.

Surgical cue generation instructions 1940 may include one or more ofincision site calculation instructions 1942, projection calculationinstructions 1944, volumetric margin calculation instructions 1946, andprojection margin calculation instructions 1948. Processor 1980 mayexecute feature incision site calculation instructions 1942, projectioncalculation instructions 1944, volumetric margin calculationinstructions 1946, and/or projection margin calculation instructions1948 to implement incision site calculator 332, projection calculator334, volumetric margin calculator 336, and/or projection margincalculator 338, respectively.

Image registration instructions 1950 may include one or more ofrigid-body transformation instructions 1952, deformable transformationinstructions 1954, gravitational transformation instructions 1955,feature location instructions 1956, and binary image generationinstructions 1957. Processor 1980 may execute rigid-body transformationinstructions 1952, deformable transformation instructions 1954,gravitational transformation instructions 1955, feature locationinstructions 1956, and/or binary image generation instructions 1957 toimplement rigid-body transformation unit 352, deformable transformationunit 354, gravity unit 355, feature locator 356, and/or binary imagegenerator 357, respectively.

In one example of operation, processor 1980 receives one or more images158 via interface 1990 and stores these to data storage 1970. Processor1980 retrieves image(s) 158 from data storage 1970 and executes locationinstructions 1930 to determine 3D spatial properties 128 based uponimage(s) 158. Processor 1980 stores 3D spatial properties 128 to datastorage 1970. Next, processor 1980 retrieves 3D spatial properties 128from data storage 1970 and executes surgical cue generation instructions1940 to determine surgical guidance cues 138 based upon 3D spatialproperties 128. Processor 1990 may store surgical guidance cues 138 todata storage 1970 and/or output surgical guidance cues 138 via interface1990.

In another example of operation, processor 1980 receives one or moreimages 358 via interface 1990 and stores these to data storage 1970.Processor 1980 retrieves image(s) 358 from data storage 1970 andexecutes image registration instructions 1950 to generate one or moreimages 158 before proceeding as outlined above in the example whereprocessor 1980 receives image(s) 158 via interface 1990.

FIG. 20 illustrates one exemplary patient-specific locator form 2000 forguiding tissue resection. As shown in FIG. 20, locator form 2000 isdesigned to guide resection of tumor 175 from patient 170. However,locator form 2000 is readily adapted for use in other tissue resectionprocedures including, but not limited to, tumor resection from otherbody parts and organs, such as the brain or the liver, and biopsyprocedures of, e.g., breast 172, muscle, or bone. Locator form 2000 isalso readily adapted to guide local delivery of markers or a therapeuticagent to patient 170.

Locator form 2000 includes features that indicate surgical guidance cues138 and/or features that enable surgeon 180 to transfer surgicalguidance cues 138 to breast 172. Locator form 2000 may help ensure goodagreement between tissue position properties of breast 172, upon whichsurgical guidance cues 138 are based, and tissue position properties ofbreast 172 when transferring surgical guidance cues 138 to breast 172.In one example of use, locator form 2000 stays on breast 172 duringresection of tumor 175. This helps maintain stable position of breast172, thus ensuring good agreement between tissue position propertiesassociated with the tissue resection procedure and tissue positionproperties upon which surgical guidance cues 138 are based. In anotherexample of use, locator form 2000 is used to transfer surgical guidancecues 138 to breast 172 in step 440 of method 400, and may be removedprior to resection surgery.

Locator form 2000 is custom made for patient 170 to fit breast 172.Locator form 2000 includes an inner surface 2090 that is adjacent tobreast 172 when locator form 2000 is placed on breast 172. Inner surface2090 is not visible in FIG. 20, since inner surface 2090 faces away fromthe observer viewing FIG. 20.

For use in other tissue resection procedures than resection of tumor175, inner surface 2090 may be adapted to fit the shape of the bodypart/organ associated with such tissue resection procedures. Forexample, inner surface 2090 may be adapted to fit the skull of a patientundergoing brain tumor resection or brain tissue biopsy.

Locator form 2000 includes fiducials that may be matched to fiducials onbreast 172 to ensure proper positioning of locator form 2000 on breast172. In the example shown in FIG. 20, locator form 2000 includes (a) afiducial 2010 that is an opening configured to fit the nipple of breast172, and (b) fiducials 2012 configured to match markings applied tobreast 172. Locator form 2000 may include more, fewer, or differentfiducials than those shown in FIG. 20, without departing from the scopehereof. In an alternative embodiment, locator form 2000 does not includefiducials 2010 and 2012, and proper placement of locator form 2000 isinstead ensured by the general shape of inner surface 2090.

For use in tissue resection procedures other than resection of tumor175, fiducials 2010/2012 may be adapted to match fiducials on the bodypart/organ associated with such tissue removal procedures.

Locator form 2000 has a cutout 2040 at the intended incision site forresection of tumor 175. In one embodiment, cutout 2040 is shaped andsized to match projection 224. Thus, cutout 2040 may function as anindicator of projection 224 and/or may enable surgeon to mark projection224 onto surface 174. In another embodiment, cutout 2040 is oversized ascompared to projection 224 but allows surgeon 180 to mark projection 224onto surface 174, for example guided by an OR navigation system asdiscussed in reference to FIGS. 4A and 4B. Although not shown in FIG.20, cutout 2040 may be a small cutout that enables surgeon 180 toaccurately mark point 222 on surface 174, without departing from thescope hereof. Furthermore, locator form 2000 may include one or moreadditional small cutouts that enable surgeon 180 to accurately mark oneor more margins of projection 224 (such as cranial margin 242, caudalmargin 244, lateral margin 246, and/or medial margin 248), and/or otherpositions along the perimeter of projection 224 or within projection224, on surface 174.

Locator form 2000 includes at least one raised needle port 2030. Eachraised needle port 2030 has a cannulation 2032 configured to accept aneedle of a syringe 2080.

In one embodiment, locator form 2000 is made of a material that is atleast partly transmissive to light, such that surgeon may see breast 172through locator form 2000. In another embodiment, a portion of locatorform 2000 is made of a material that is at least partly transmissive tolight, such that surgeon may see at least a portion breast 172 throughthis light-transmissive portion of locator form 2000. In anotherembodiment, locator form 2000 is tessellated (for example according to aVoronoi pattern) to reduce the amount of material required to producelocation form 2000 without compromising the structural integrity oraccuracy of locator form 2000.

In certain embodiments, locator form 2000 is extended to further includea form portion 2050 that is configured to match the shape of portions ofpatient 170 near breast 172. In one example, form portion 2050 isconfigured to match the shape of more rigid anatomical structures ofpatient 170 such as at least a portion of the rib cage and/or sternum ofpatient 170. Form portion 2050 may improve the accuracy of alignment oflocator form 2000 with respect to patient 170.

Without departing from the scope hereof, locator form 2000 may includeonly some of the features shown in FIG. 20. For example, someembodiments of locator form 2000 such as those for use with roboticinjectors may omit raised needle ports 2030.

Although not shown in FIG. 20, locator form 2000 may include, or beconfigured to attach to, one or more restraining mechanisms to securelocator form 2000 to breast 172 after aligning locator form 2000 tobreast 172 using fiducials 2010 and 2012. Exemplary restrainingmechanisms include elastic and non-elastic straps.

In an alternative use scenario, locator form 2000 is placed on patient170 to place a marker or therapeutic agent in local tissue of patient170 through raised needle port(s) 2030. For example, locator form 2000may be used to inject radioactive seeds into an organ of patient 170,such as the prostate or breast 172 of patient 170.

FIG. 21A further illustrates the functionality of exemplary raisedneedle ports 2030. FIG. 21A is best viewed together with FIG. 20. Byvirtue of cannulation 2032, raised needle port 2030 defines a direction2110 to a location 2122 within breast 172. For example, direction 2110may lead to a location 2122 associated with tumor 175, such as a marginof tumor 175, or other location within or on the perimeter of tumor 175,wherein this margin, or other interior or perimeter location, may beextended from tumor 175 by a volumetric safety margin 282 as discussedin reference to FIG. 2C. Raised needle port 2030 is configured topassively guide a needle of known length to location 2122. Surgeon 180may use raised needle port to inject a dye into tumor 175, either tomark all of tumor 175 or to mark a margin (or other interior orperimeter location) of tumor 175. As an alternative to dye, surgeon 180may use other physical markers including, but not limited to,biocompatible self-powered light-emitting diodes, a radioactive seed,RF-sensitive markers (such as those disclosed in U.S. Pat. No.8,892,185), cauterization, and other markers known in the art. Surgeon180 may use raised needle port 2030 to insert an associated deliverydevice into breast 172, such as those disclosed in U.S. Pat. No.8,892,185 or other devices known in the art. In one example, surgeon 180uses raised needle port 2030 and an associated delivery device to injectradioactive seed(s) or RF-sensitive markers to the centroid 212 of tumor210. Marking of tumor 175 may provide a surgical guidance cue to aidassessment of resection of tumor 175 and, in turn, help ensure that allcancerous tissue is removed from breast 172. Syringe 2080 (or analternative delivery device) may be configured to inject a meteredvolume of dye (or an alternative marker) to avoid excessive diffusion ofdye (or alternative marker) to a larger region than intended.

In an alternative embodiment, one raised needle port 2030 is placed atthe incision site (for example at point 222) for resection of tumor 175and is used to place a hook wire at location 2122, such that a distalend of the hook wire is at location 2122 and a proximal end of the hookwire exits surface 174 at the incision site. In this embodiment, thehook wire functions as a surgical guidance cue, and this raised needleport 2030 enables accurate placement of the hook wire to provide thissurgical guidance cue with high accuracy. In one example of thisembodiment, this raised needle port 2030 directs the hook wire toanterior margin 214. In another example of this embodiment, this raisedneedle port 2030 directs the hook wire to the center of tumor 175 (at ornear centroid 212). In yet another example of this embodiment, thisraised needle port 2030 directs the hook wire to a surface of tumor 175such as an anterior or posterior surface of tumor 175. In this example,this raised needle port 2030 may direct the hook wire through the centerof tumor 175 to the posterior surface of tumor 175, for example toposterior margin 216. In a further example of this embodiment, thisraised needle port 2030 directs the hook wire to a location within tumor175. Without departing from the scope hereof, this raised needle port2030 may be placed away from the incision point (point 222, for example)and be oriented to direct the hook wire to tumor 175 (for example, toanterior margin 214, the center of tumor 175, a posterior surface oftumor 175, an anterior surface of tumor 175, or another location on orwithin tumor 175) from a location that is different from the incisionpoint.

In certain embodiments, locator form 2000 includes (a) one raised needleport 2030 configured to passively guide a hook wire to tumor 175 asdiscussed above, and (b) one or more raised needle ports 2030 eachconfigured to passively guide a syringe 2080 (or alternative deliverydevice) to tumor 175 as discussed above.

The functionality of raised needle port 2030 (or alternative fiducial)is readily extended to marking of tumors in other body parts/organ, suchas the brain or liver, or alternatively to local delivery of atherapeutic agent. Furthermore, raised needle port 2030 may beconfigured to accept a biopsy needle and thus function as a surgicalguidance cue for a biopsy procedure of, e.g., breast 172, muscle, orbone.

FIGS. 21B and 21C illustrate one exemplary two-part raised needle port2150 that targets two different positions within breast 172. Two-partraised needle port 2150 is an embodiment of raised needle port 2030.Two-part raised needle port 2150 includes a base portion 2152, closestto inner surface 2090, and a removable spacer portion 2154 furthest frominner surface 2090. Cannulation 2032 extends through both of baseportion 2152 and spacer portion 2154 such that a needle may be insertedinto breast 172 through cannulation 2032. FIG. 21B shows two-part raisedneedle port 2150 with spacer portion 2154 and FIG. 21C shows two-partraised needle port 2150 without spacer portion 2154. FIGS. 21B and 21Care best viewed together.

FIGS. 21B and 21C illustrate two-part raised needle port 2150 asimplemented in locator form 2000. For clarity of illustration, FIGS. 21Band 21C do not show other parts of locator form 2000. In the scenarioshown in FIGS. 21B and 21C, two-part raised needle port 2150 isconfigured to (a) direct a syringe needle 2182 of syringe 2080 throughthe incision site (point 222) to anterior margin 214 when spacer portion2154 is in place to inject dye at anterior margin 214, and (b) direct ahook wire insertion needle 2184 of a hook wire insertion device 2180through the incision site (point 222, for example) to posterior margin216 when spacer portion 2154 is removed from two-part raised needle port2150 to place a hook wire at posterior margin 216 with the hook wirepassing through the incision point. However, without departing from thescope hereof, two-part raised needle port 2150 may be used to target anytwo positions located at two different depths within breast 172 alongthe axis defined by cannulation 2032. Furthermore, two-part raisedneedle port 2150 may be used to target two such positions for injectionof dye or other markers to both of these positions.

In one embodiment, two-part raised needle port 2150 is configured foruse with two needles of the same length, and spacer portion 2154 haslength 2190 equal to the distance 2192 between the two targetedpositions. In another embodiment, two-part raised needle port 2150 isconfigured for use with two needles of different lengths, and spacerportion 2154 has length 2190 that is distance 2192 corrected for thedifferent needle lengths. Two-part raised needle port may accept needlesof a variety of calibers such that the two different needles may be ofdifferent caliber.

The functionality of two-part raised needle port 2150 is readilyextended to marking of tumors in other body parts/organ, such as thebrain or liver, or to local delivery of a therapeutic agent.Furthermore, two-part raised needle port 2150 may be configured toaccept a biopsy needle and thus function as a surgical guidance cue fora biopsy procedure of, e.g., breast 172, muscle, or bone.

FIG. 22 illustrates one exemplary method 2200 for guiding resection oftumor 175 using a locator form such as locator form 2000.

In a step 2210, the locator form is placed on breast 172. Step 2210 mayinclude a step 2212 of matching fiducials on the locator form withfiducials on breast 172. In one example of step 2210, locator form 2000is placed on breast 172 with inner surface 2090 facing surface 174.Optionally, fiducial 2010 is placed at the location of the nipple ofbreast 172 and fiducials 2012 are matched to fiducials on surface 174.

In a step 2220, surgical guidance cues 138 are transferred to breast 172using features of the locator form. In one example of step 2220,surgical guidance cues 138 are transferred to breast 172 using featuresof locator form 2000.

Step 2220 may include a step 2222 of marking the incision site, forresection of tumor 175, at least in part based upon a feature of thelocator form. In one example of step 2222, point 222 is marked onsurface 174 based upon cutout 2040 and optionally further guided byother OR navigation systems, such as those discussed in reference toFIG. 1. In another example of step 2222, point 222 and one or more ofcranial margin 242, caudal margin 244, lateral margin 246, and/or medialmargin 248 (and/or other tumor perimeter location) are marked on surface174 based upon respectively smaller cutouts of locator form 2000, asdiscussed in reference to FIG. 20. In yet another example of step 2222,the perimeter of projection 224 is marked on surface 174. In thisexample, cutout 2040 may match the shape and location of the perimeterof projection 224.

Step 2220 may include a step 2224 of marking projection 224 of tumor 175onto surface 174 as guided by a feature of the locator form. In oneexample of step 2224, cutout 2040 of locator form 2000 matchesprojection 224, and projection 224 is marked on surface 174 as guided bycutout 2040.

Optionally, step 2220 includes a step 2226 of injecting dye (or othermarker) into tumor 175 through a needle port of the locator form. In oneexample of step 2226, dye (or other marker) is injected into tumor 175to mark the volume of tumor 175 or a margin of tumor 175, such asanterior margin 214 or posterior margin 216, (and/or one or more otherlocations on the perimeter of tumor 175) using a syringe 2080 guided bycannulation 2032 of a corresponding raised needle port 2030. Step 2226may utilize a plurality of raised needle ports 2030 to mark differentportions of tumor 175. Alternatively, step 2220 may include a step 2228of inserting a hook wire into breast 172, for example as discussed inreference to FIGS. 21A-C.

Optionally, the locator form is removed from breast 172 in a step 2230subsequent to step 2220.

In an embodiment, method 2200 includes a step 2260 of performingresection surgery to remove tumor 175. In one implementation, step 2260is performed after removal of the locator form in step 2230. This mayimprove access to breast 172, as compared to when the locator form is inplace on breast 172. In another implementation, step 2260 is performedwhile the locator form is in place on breast 172. This may help maintainthe tissue positions properties, upon which features of the locator formare based. In one example, a biopsy procedure is performed with thelocator form in place on breast 172. In another example, an ablativetumor resection surgery, such as RF ablation, cryogenic ablation, orhigh-intensity focused ultrasound ablation, is performed with thelocator form in place on breast 172.

Method 2200 is readily extended to tumor resection from other bodyparts/organs of patient 170. Additionally, method 2200 may be extendedto other tissue resection procedures, biopsy procedures, or therapeuticagent delivery procedures, without departing from the scope hereof. Inone such example, method 2200 is implemented with step 2226 (or step2228), whereafter, in step 2260, a biopsy needle is inserted into breast172 to the location marked in step 2226. In another example, a biopsyneedle is inserted into breast 172 in step 2228.

FIG. 23 illustrates a patient-specific locator form 2300 that furtherincludes a material model of tumor 175. Patient-specific locator form2300 is an extension of patient-specific locator form 2000, whichfurther includes a material model 2310 of tumor 175 and a connecting rod2330. Connecting rod 2330 is configured to cooperate with locator form2000 to place material model 2310 in a location that matches thelocation of tumor 175 relative to surface 174. Locator form 2300 helpssurgeon 180 visualize the geometry associated with resection surgery toremove tumor 175.

Locator form 2300 is readily extended to other tissue resectionprocedures by modifying locator form 2000 as discussed above andreplacing material model 2310 with a material model of the tissue to beremoved in such other tissue resection procedures.

FIG. 24 illustrates one exemplary system 2400 for manufacturingpatient-specific locator form 2000. System 2400 is an extension ofsystem 100, which includes model generator 340 and further includes a 3Dprinter 2440 for additively manufacturing locator form 2000 based upondetermined surgical guidance cues 138 and model 348.

Model generator 340 communicates a 3D surface model 348 of at least aportion of surface 174 to 3D printer 2440. Based upon this 3D surfacemodel 348, 3D printer 2440 additively manufactures locator form 2000such that inner surface 2090 substantially matches surface 174. 3Dprinter 2440 additively manufactures locator form 2000 with features(such as cutout 2040) that indicate one or more surgical guidance cues138 and/or features (such as raised needle port(s) 2030 or alternativeneedle fiducials) that may be utilized to transfer one or more surgicalguidance cues 138 to breast 172. In certain implementations, modelgenerator 340 further communicates a volumetric model 348 of tumor 175to 3D printer 2440, and 3D printer 2440 further manufactures materialmodel 2310 and connecting rod 2330.

Without departing from the scope hereof, tumor 175 may be other localtissue than a breast tumor, for example a tumor in a differentorgan/body part of patient 170, local tissue from which a biopsy must beperformed, or local tissue to which a therapeutic agent must bedelivered, and system 2400 may produce a locator form 2000 that matchesan associated surface of patient 170. Also without departing from thescope hereof, system 2400 may utilize prone images of breast 172 asopposed to supine images 158, so as to produce a locator form that fitsbreast 172 in prone position.

FIG. 25 illustrates one exemplary system 2500 for manufacturing aplurality of patient-specific locator forms 2000 for a respectiveplurality of patients 170 based upon supine images 158. System 2500 islocated at a central manufacturing facility 2580, while patients 170 andassociated imaging modules 150 may be located at one or more remotefacilities 2590. Remote facilities 2590 are, for example, one or morehospitals or operating/imaging facilities remote from centralmanufacturing facility 2580. Each imaging module 150 of remotefacilities 2590 communicates at least one supine image 158 of breast172, of respective patient 170, via a data connection 2570 to locationmodule 120 and model generator 340 at central manufacturing facility2580. Based upon supine images 158, location module 120 and surgical cuegenerator 130 generates surgical guidance cues 138 for each patient 170,and model generator 340 generates model 348 for each patient 170, asdiscussed in reference to FIG. 24 for a single patient 170. Usingsurgical guidance cues 138 for each patient 170 and model 348 for eachpatient 170, one or more 3D printers 2440 additively manufactures alocator form 2000 specific to each patient 170, as discussed inreference to FIG. 24 for a single patient 170.

Each data connection 2570 may be a Digital Imaging and Communications inMedicine (DICOM) connection.

Although not shown in FIG. 25, system 2500 may include imageregistration module 350 such that system 2500 may receive images 358instead of supine images 158, without departing from the scope hereof.System 2500 is readily extended to production of locator forms forguiding other tissue resection procedures than breast tumor resection,without departing from the scope hereof.

FIG. 26 illustrates one exemplary method 2600 for producingpatient-specific locator form 2000 for breast 172. In a step 2630, 3Dprinter 2440 additively manufactures locator form 2000 based upon (a) a3D surface model 348 of surface 174 and (b) surgical guidance cues 138.Step 2630 includes a step 2632 of additively manufacturing locator form2000 to form inner surface 2090, based upon 3D surface model 348. Step2630 further includes a step 2634 of additively manufacturing locatorform 2000 to form features that indicate surgical guidance cues 138and/or features that enable transfer of surgical guidance cues to breast172. In one example of step 2634, 3D printer 2440 forms one or morefeatures such as features discussed in reference to FIGS. 20 and 21.

In an embodiment, method 2600 further includes a step 2610 of generating3D surface model 348 of surface 174. In one example of step 2610, modelgenerator 340 generates 3D surface model 348 of surface 174.

In an embodiment, method 2600 further includes a step 2620 of generatingsurgical guidance cues 138. In one example of step 2610, surgical cuegenerator 130 generates surgical guidance cues 138. Without departingfrom the scope hereof, system 2400 (and also system 2500) may implementfunctionality that specifies features of locator form 2000 based uponsurgical guidance cues 138.

Method 2600 is readily extended to production of locator forms forguiding other tissue resection procedures than breast tumor resection,without departing from the scope hereof. Likewise, method 2600 mayutilize a 3D surface model 348 and surgical guidance cues 138 generatedfrom prone images of breast 172 as opposed to supine images 158, so asto produce a locator form that fits breast 172 in prone position.

FIG. 27 illustrates one exemplary method 2700 for producing materialmodel 2310 and connecting rod 2330. In a step 2720, 3D printer 2440additively manufactures material model 2310 and connecting rod 2330based upon a volumetric model 348 of breast 172. This volumetric model348 includes a model of tumor 175 and indicates position of tumor 175relative to surface 174. Optionally, method 2700 includes a step 2710wherein model generator 340 generates this volumetric model 348.

Without departing from the scope hereof, method 2700 may utilize a 3Dsurface model 348 generated from prone images of breast 172 as opposedto supine images 158, so as to produce material model 2310 andconnecting rod 2330 indicative of breast 172 in prone position.

Methods 2600 and 2700 may be combined to produce patient-specificlocator form 2300.

FIG. 28 illustrates one exemplary navigation system 2800 for guidingresection of tumor 175 from breast 172 of patient 170 with the aid ofsurgical guidance cues 138. Navigation system 2800 implementsvisualization module 140.

Navigation system 2800 includes a display 2810 that displays surgicalguidance cues 138. Navigation system 2800 further includes a trackingstylus 2820 and a tracking reference 2830. Although not shown in FIG.28, navigation system 2800 also includes a computer module that (a)processes tracking data generated by tracking stylus 2820 and trackingreference 2830 to determine the position and, optionally, orientation oftracking stylus 2820, and (b) processes surgical guidance cues 138(generated by surgical cue generator 130 in step 420 of method 400) andone or more models (generated by model generator 340 in step 430 ofmethod 400) to visualize surgical guidance cues 138 and tracking stylus2820 on display 2810. Navigation system 2800 uses tracking reference2830 to reference the position and, optionally, orientation of trackingstylus 2820 to the position and orientation of patient 170.

In the scenario shown in FIG. 28, display 2810 displays a model ofbreast 172, which includes a model 2812 of breast surface 174 and amodel 2814 of tumor 175. Displays 2810 shows model 2812 and model 2814in proper positional relationships with each other to provide avisualization of breast 172 to surgeon 180. Display 2810 also shows amodel 2816 of tracking stylus 2820 overlaid on models 2812 and 2814 toprovide surgeon 180 with a visualization of the actual position and,optionally, orientation of tracking stylus 2820 with respect to featuresin models 2812 and 2814. Although not visible in FIG. 28, display 2810shows surgical guidance cues 138 overlaid on models 2812 and 2814.

Guided by display 2810, surgeon 180 moves tracking stylus 2820 to marksurgical guidance cues 138 on surface 174, thus transferring one or moresurgical guidance cues 138 to surface 174. In the example shown in FIG.28, surgeon 180 uses tracking stylus 2820 to mark, on surface 174,incision site 2850 and projection 2840 of tumor 175 onto surface 174 (asdiscussed in reference to FIGS. 5A-C).

Without departing from the scope hereof, tracking stylus 2820 may bereplaced by, or operated in conjunction with, a tracking syringe (orother delivery device). Navigation system 2800 tracks the position andorientation of the tracking syringe and overlays the position andorientation of the tracking syringe on models 2812 and 2814 on display2810. The tracking syringe may have three non-collinear tracking nodesthat allow navigation system 2800 to determine the position andorientation of the tracking syringe.

Fiducial markers 2860 may be placed on surface 174 to aid registrationof images 358 to produce surgical guidance cues 138. For clarity ofillustration, not all fiducial markers 2860 are labeled in FIG. 28.

Navigation system 2800 is readily extended to tumor resection from otherbody parts/organs of patient 170. Additionally, navigation system 2800may be extended to other tissue resection procedures, biopsy procedures,or therapeutic agent delivery procedures, without departing from thescope hereof.

FIG. 29 illustrates one exemplary method 2900 for visualizing surgicalguidance cues 138 using a navigation system. Method 2900 is anembodiment of step 430 of method 400.

A step 2920 displays a volumetric model of breast 172. The volumetricmodel includes a model of surface 174 and a model of tumor 175, whereinthe model of surface 174 and the model of tumor 175 are shown in properpositional relationships to each other, thus visualizing breast 172. Themodel of surface 174 is shown as being semitransparent such that themodel of tumor 175 is visible on the display. In one example of step2920, navigation system 2800 displays models 2812 and 2814, as generatedin step 430 of method 400, on display 2810.

A step 2930 overlays surgical guidance cues 138 on the models displayedin step 2920. In one example of step 2930, navigation system 2800overlays surgical guidance cues 138 on models 2812 and 2814 on display2810. Without departing from the scope hereof, steps 2920 and 2930 maybe performed in the reverse order or in parallel.

Optionally, steps 2920 and 2930 are preceded by a step 2910 oftransforming to the coordinate system associated with the navigationsystem (a) models generated in step 430 of method 400 and displayed instep 2920 and (b) surgical guidance cues 138 generated in step 420 ofmethod 400 and displayed in step 2930. Step 2910 may transform thesesurgical guidance cues 138 and models to a coordinate system that isreferenced to a tracking reference of the navigation system. In oneexample of step 2910, navigation system 2800 transforms surgicalguidance cues 138 and models of breast 172 to a coordinate systemreferenced to tracking reference 2830.

A step 2940 tracks the position and, optionally orientation, of one ormore tracking devices tracked with respect to patient 170. Step 2940overlays the position and, optionally, orientation of these trackingdevices on the models displayed in step 2920. In one example of step2940, navigation system 2800 tracks the position and, optionally,orientation of tracking stylus 2820 and surgeon 180 uses tracking stylusto mark one or more of the surgical guidance cues 138, for example theincision site (e.g., point 222), on surface 174.

In certain embodiments, method 2900 further includes one or both ofsteps 2950 and 2960. In step 2950, surgeon 180 uses a tracked syringe(or other delivery device) to inject dye (or other marker, or atherapeutic agent) into breast 172 as guided by the models generated instep 2920 and/or the surgical guidance cues 138 generated in step 2930.In one example of step 2950, surgeon 180 uses a tracked syringe toinject dye into breast 172 at the location of each of one or moremargins of tumor 175, such as anterior margin 214 and posterior margin216, (and/or at one or more other locations on the perimeter of tumor175) as guided by (a) navigation system 2800 and (b) the modelsgenerated in step 2920 and/or the surgical guidance cues 138 generatedin step 2930. In step 2960, a robotically controlled syringe (or otherdelivery device) injects dye (or other marker, or a therapeutic agent)into breast 172 as guided by the models generated in step 2920 and/orthe surgical guidance cues 138 generated in step 2930. In one example ofstep 2960, surgeon 180 uses a tracked syringe to inject dye into breast172 at the location of each of one or more margins of tumor 175, such asanterior margin 214 and posterior margin 216, (and/or at one or moreother locations on the perimeter of tumor 175) as guided by (a)navigation system 2800 and (b) the models generated in step 2920 and/orthe surgical guidance cues 138 generated in step 2930.

Method 2900 is readily extended to tumor resection from other bodyparts/organs of patient 170. Additionally, method 2900 may be extendedto other tissue resection procedures, biopsy procedures, or therapeuticagent delivery procedures, without departing from the scope hereof.

FIG. 30 illustrates one exemplary method 3000 for transferring surgicalguidance cues 138 to breast 172 using a navigation system and trackingdevices. Method 3000 is an embodiment of step 440 of method 400.

In a step 3010, method 3000 uses a tracking stylus to transfer surgicalguidance cues 138 to surface 174. In one example of step 3010, surgeon180 operates tracking stylus 2820. Based upon a visualization generatedby method 2900, surgeon 180 marks surface 174 to indicate one or moresurgical guidance cues 138 on surface 174. For example, surgeon 180 mayindicate incision site 2850 and projection 2840.

Optionally, method 3000 includes a step 3020 of transferring a surgicalguidance cue 138 to the interior of breast 172. In step 3020, surgeon180 uses a tracked syringe to inject dye into breast 172 to mark one ormore surgical guidance cues 138 such as cranial margin 242, caudalmargin 244, lateral margin 246, medial margin 248, and/or otherlocation(s) on the perimeter of tumor 175. Surgeon 180 places thetracked syringe according to a visualization of the position andorientation of the tracked syringe relative to a model of breast 172. Inone example of step 3020, surgeon 180 uses navigation system 2800implemented with a tracking syringe to place the needle tip of thetracking syringe at a location within breast 172. Surgeon 180 places theneedle tip of the tracking syringe as guided by a visualizationdisplayed on display 2810. When the needle tip is at the desiredlocation within breast 172, surgeon 180 injects dye into breast 172 viathe syringe.

Method 3000 is readily extended to tumor resection from other bodyparts/organs of patient 170. Additionally, method 3000 may be extendedto other tissue resection procedures, biopsy procedures, or therapeuticagent delivery procedures, without departing from the scope hereof.

FIG. 31 illustrates one exemplary method 3100 for automaticallytransferring surgical guidance cues 138 to breast 172 using a roboticsystem. Method 3100 is an embodiment of step 440 of method 400.

In a step 3110, method 3100 uses a robotically controlled stylus toautomatically transfer one or more surgical guidance cues 138 to surface174. The robotically controlled stylus is operated according to surgicalguidance cues 138 and models generated in steps 420 and 430 of method400.

In an optional step 3120, method 3100 uses a robotically controlledsyringe to automatically transfer one or more surgical guidance cues 138to the interior of breast 172 by injecting dye into breast 172 at one ormore locations such as one or more of cranial margin 242, caudal margin244, lateral margin 246, medial margin 248, and/or other location(s) onthe perimeter of tumor 175.

Method 3100 is readily extended to tumor resection from other bodyparts/organs of patient 170. Additionally, method 3100 may be extendedto other tissue resection procedures, biopsy procedures, or therapeuticagent delivery procedures, without departing from the scope hereof.

Without departing from the scope hereof, methods 3000 and 3100 may becombined. For example, surgical guidance cues 138 may be transferred tosurface 174 using a tracking stylus, as discussed in reference to step3010 while surgical guidance cues 138 are transferred to the interior ofbreast 172 using a robotically controlled syringe as discussed inreference to step 3120.

FIG. 32 illustrates one exemplary method 3200 for guiding resection of atumor 175, which does not require image registration. Method 3200 is anembodiment of method 400 and may be performed by system 100.

In a step 3210, method 3200 generates at least one supine image 158 ofbreast 172 in a supine position at least substantially the same as theposition used during resection surgery. Step 3210 is an embodiment ofstep 410. In a step 3220, method 3200 performs method 400 without step410. In an optional step 3230, surgeon 180 performs the tissue resectionprocedure to resect tumor 175, while utilizing surgical guidance cues138 determined in step 3220.

In one example of method 3200, breast 172 does not exhibit significanttissue displacement between image capture in step 3210 and tissueresection surgery subsequent to step 3220. In another example of method3200, a locator form 2000 is manufactured according to image(s) 158generated in step 3210, and locator form 2000 ensures that, even iftissue displacement takes place between image capture in step 3210 andtissue resection surgery subsequent to step 3220, the tissue positioningof breast 172 is restored to the tissue positioning at the time of imagecapture in step 3210.

FIG. 33 illustrates one exemplary method 3300 for guiding resection of atumor 175 based upon preoperative supine volumetric image(s) 358 andsupine 3D surface image(s) 358 representative of the supine positionused during resection surgery. Method 3300 is an embodiment of method400 and may be performed by system 100.

In a step 3310, method 3300 generates at least one preoperativevolumetric image 358 of breast 172 in a first supine position. In a step3320, method 3300 generates at least one supine 3D surface image 358 ofsurface 174 with breast 172 in the same position as associated with theresection surgery. The first supine position is different from theresection-associated position, such that there is some tissuedisplacement between steps 3310 and 3320. In a step 3330, method 3300performs method 400 with steps 412 and 414 based upon images captured insteps 3310 and 3320. Steps 3310, 3320, and 3330 together form anembodiment of step 410. In an optional step 3340, surgeon 180 performsthe tissue resection procedure to resect tumor 175, while utilizingsurgical guidance cues 138 determined in step 3330.

FIG. 34 illustrates one exemplary method 3400 for guiding resection of atumor 175, which extracts both volumetric and surface data from the samepreoperative volumetric image(s). Method 3400 thus eliminates the needto co-register volumetric and surface images. Method 3400 is anembodiment of method 400 and may be performed by an embodiment of system100 that implements volumetric imager 152. Method 3400 does not requiresurface imager 154. Consequently, method 3400 eliminates the need foracquiring separate surface image(s), and may provide improved accuracyby eliminating any influence from potential co-registration errorsbetween separately acquired volumetric and surface images.

In a step 3410, method 3400 generates at least one preoperativevolumetric image 358 of breast 172 in supine position. Optionally, step3410 implements a step 3412 of generating a magneto resonance image ofbreast 172 in supine position, which may be a single magneto resonanceimage or a plurality of substantially co-registered magneto resonanceimages.

In a step 3420, method 3400 extracts a 3D surface image 358 of breast172 from the preoperative volumetric image 358 generated in step 3410.This 3D surface image 358 is inherently co-registered with thepreoperative volumetric image 358 generated in step 3410.

In a step 3430, method 3400 performs method 400 without step 410. Step3430 utilizes (a) volumetric image data from the preoperative volumetricimage 358 generated in step 3410 and (b) 3D surface image 358 generatedin step 3420 and inherently co-registered with the volumetric imagedata.

In an optional step 3440, surgeon 180 performs the tissue resectionprocedure to resect tumor 175, while utilizing surgical guidance cues138 determined in step 3330 based upon data obtained from thepreoperative volumetric image 358 generated in step 3410.

In an embodiment, step 3420 includes steps 3422 and 3424. Step 3422segments the tissue volume in the preoperative volumetric image 358generated in step 3410, that is, step 3422 identifies tissue versus airin preoperative volumetric image 358 and extracts the tissue volume frompreoperative volumetric image 358. Step 3424 processes the tissuevolume, segmented in step 3422, to generate a 3D surface model 348 ofbreast 175. In one embodiment, step 3424 includes sequential steps 3426and 3427, and optionally also a step 3428. Step 3426 at least partlycleans up the tissue volume, generated in step 3422, for motion and/orsignal artifacts to generate a cleaned-up tissue volume. Step 3427generates a 3D surface model of the cleaned-up tissue volume of step3426. Step 3427 may utilize a tessellation algorithm or other methodknown in the art. Optional step 3428 processes the 3D surface modelgenerated in step 3427 to generate a connected (“water-tight”) 3Dsurface model, if the 3D surface model generated in step 3427 is notfully connected. Step 3428 may utilize a Poisson surface reconstructionalgorithm, for example as known in the art. Step 3424 may furtherinclude a step 3429 of selecting a relevant portion of either the 3Dsurface model generated in step 3427 or the connected 3D surface modelgenerated in step 3428. Step 3429 may include loading the 3D surfacemodel generated in step 3428 or 3429 into a mesh editing software tomanually select and retain useful breast surface while eliminating otherunwanted structures. In one example of method 3400 implementing step3429, step 3430 implements step 438, and the 3D surface model portionselected in step 3429 is the portion intended to be matched with apatient-specific locator form generated in step 438. Without departingfrom the scope hereof, step 3424 may include step 3427 but not step3426.

Without departing from the scope hereof, step 3410 may be replaced by astep of receiving preoperative volumetric image(s) 358 from an externalsystem, such as an image repository or a third-party imaging system.

FIG. 35 shows exemplary image data illustrating image processing by oneembodiment of method 3400. In this example, step 3410 generates orreceives a high-resolution magneto resonance image 3500 of breast 175,for example with 1-millimeter slice spacing and having a field of viewsufficient to include rigid anatomical areas around breast (such as aportion of the sternum and/or portions below the infra mammary fold).Step 3422 segments a tissue volume 3510 from magneto resonance image3500. Step 3427 (optionally in cooperation with step 3426) generates a3D surface model 3520. Step 3428 applies a Poisson surfacereconstruction algorithm to 3D surface model 3520 to generate aconnected 3D surface model 3530. Step 3429 selects a portion ofconnected 3D surface model 3530 to which a matching patient-specificlocator form may be produced in step 438.

FIG. 36 illustrates one exemplary method 3600 for guiding resection of atumor 175, using a patient-specific locator form manufactured based uponvolumetric and surface data obtained from the same preoperativevolumetric image(s) 358. Method 3600 is an embodiment of method 400 andof method 3400. Method 3600 includes an embodiment of method 2200.Method 3600 may be performed by an embodiment of system 100 thatimplements volumetric imager 152 or by an embodiment of system 2400 thatimplements volumetric imager 152. Method 3600 does not require surfaceimager 154.

In a step 3610, method 3600 generates at least one preoperativevolumetric image 358 of breast 175 in supine position. Step 3610 is anembodiment of step 3410, which includes a step 3612 of utilizing one ormore fiducial markers on surface 174 of breast 175 such that thepreoperative volumetric image(s) 358 indicates the position of the oneor more fiducial markers.

In a step 3620, method 3600 performs step 3420 of method 3400 to extracta 3D surface image 358 of breast 175, wherein the 3D surface image 358shows the position of the one or more fiducial markers.

In a step 3630, method 3600 performs steps 420, 430, and 438 (andoptionally step 436) of method 400 to generate a patient-specificlocator form that fits surface 174 of breast 175 and indicates bothsurgical guidance cues 138 and the positions of the fiducial marker(s)of step 3612.

In certain embodiments, method 3600 includes a step 3640, wherein method3600 performs steps 2210 (including step 2212) and 2220 of method 2000to place the patient-specific locator form on breast 175 and transfersurgical guidance cues 138 to breast 175 based upon features of thepatient-specific locator form.

Method 3600 may further include a step 3650 of performing step 2260 ofmethod 2200 (optionally preceded by step 2230 of method 2200) to performthe tissue resection surgery with the aid of surgical guidance cues 138of step 3640.

FIG. 37 illustrates one exemplary method 3700 for guiding resection of atumor 175 based upon preoperative prone volumetric image(s) 358 andsupine 3D surface image(s) 358 representative of the supine positionused during resection surgery. Method 3700 is an embodiment of method400 and may be performed by system 100.

In a step 3710, method 3700 generates at least one preoperativevolumetric image 358 of breast 172 in prone position. In a step 3720,method 3700 generates at least one supine 3D surface image 358 ofsurface 174 with breast 172 in the same position as associated with theresection surgery. In a step 3730, method 3700 performs method 400 withstep 412 based upon the images generated in steps 3710 and 3720 and withstep 412 implementing FEM method 1500 to account for the significanttissue displacement between step 3710 and step 3720. Steps 3710, 3720,and 3730 together form an embodiment of step 410. In an optional step3740, surgeon 180 performs the tissue resection procedure to resecttumor 175, while utilizing surgical guidance cues 138 determined in step3730.

FIG. 38 illustrates one exemplary method 3800 for guiding resection of atumor 175, wherein surgical guidance cues 138 are transferred to breast172 preoperatively, for example in the imaging suite. Method 3800 is anembodiment of method 400 and may be performed by system 100.

In a step 3810, method 3800 generates at least one preoperative supineimage 158 or 358 of breast 172. Step 3810 is an embodiment of step 410.In a step 3820, method 3800 performs method 400 without step 410 andwith steps 430 and 440. Step 3820 is performed preoperatively withbreast 172 in the position used during step 3810. Step 3820 may beperformed in the imaging suite. Surgical guidance cues 138 aretransferred to breast 172 while breast 172 is in the position used forimage capture in step 3810. In an optional step 3830, surgeon 180performs the tissue resection procedure to resect tumor 175, whileutilizing surgical guidance cues 138 transferred to breast 172 in step3820. Tissue displacement between steps 3820 and 3830 does not adverselyimpact the accuracy of surgical guidance cues 138 transferred to breast172.

Each of methods 3200, 3300, 3400, 3600, 3700, and 3800 may, for example,be extended to guide tumor resection from other body parts and organs,such as the brain or the liver, as well as guide biopsy procedures of,e.g., muscle or bone. As such, As such, tumor 175 may be generalized tolocal tissue of patient 170, breast 172 may be generalized to a portionof patient 170 associated with the resection surgery, surface 174 may begeneralized to be a surface of patient 170 near the local tissue andincluding the incision site for removing the local tissue, and supineimage 158 may be generalized to an image of the portion of patient 170associated with the tissue resection procedure positioned as during thetissue resection procedure. Each of methods 3200, 3300, 3400, and 3500may further be used to guide local delivery of markers or a therapeuticagent to patient 170.

Combinations of Features

Features described above as well as those claimed below may be combinedin various ways without departing from the scope hereof. For example, itwill be appreciated that aspects of one system, or method, for guidingtissue resection, described herein, may incorporate or swap features ofanother system, or method, for guiding tissue resection, describedherein. The following examples illustrate some possible, non-limitingcombinations of embodiments described above. It should be clear thatmany other changes and modifications may be made to the systems andmethods herein without departing from the spirit and scope of thisinvention:

(A1) A method for guiding resection of local tissue from a patient mayinclude (a) generating at least one first image, of the patient,including image of the local tissue and image of at least a portion ofsurface of the patient, (b) automatically determining, at least in partbased upon the first image, a plurality of surgical guidance cuesindicating three-dimensional spatial properties associated with thelocal tissue, and (c) generating a visualization of the surgicalguidance cues relative to the surface.

(A2) In the method denoted as (A1), the step of generating at least onefirst image may include (a) capturing at least one volumetric image, ofthe patient, including the image of the local tissue, and (b) capturingat least one surface image, of the patient, including the image of atleast a portion of surface of the patient.

(A3) In the method denoted as (A1), the step of generating at least onefirst image may include capturing at least one volumetric image, ofportion of the patient, including the image of a local tissue and theimage of at least a portion of surface of the patient.

(A4) In the method denoted as (A1), the step of generating at least onefirst image may include (a) capturing at least one second image of thepatient when the patient is in a second position, the second imageincluding a second-position image of the local tissue andsecond-position image of at least a portion of the surface, (b)capturing at least one surface image of the patient when the localtissue and the portion of the surface is in a first position associatedwith the resection, and (c) transforming the second image to registerthe surface as captured in the second image to the surface as capturedin the surface image, to produce the first image.

(A5) In the method denoted as (A4), the local tissue may be a tumor in abreast of the patient, each of the first position and the secondposition may be a supine position, and the step of transforming mayinclude producing the first image by performing a rigid-bodytransformation of the second image to register the surface as capturedin the second image to the surface as captured in the surface image.

(A6) In the method denoted as (A4), the local tissue may be a tumor in abreast of the patient, the first position may be a supine position, thesecond position being a prone position, and the step of transforming mayinclude producing the first image by deformably transforming of thesecond image to register the surface as captured in the second image tothe surface as captured in the surface image while accounting for tissuedisplacement of the breast between the prone position and the supineposition.

(A7) In any of the methods denoted as (A1) through (A6), the step ofgenerating a visualization may include generating a model of thesurface, and superimposing at least one of the surgical guidance cues onthe model of the surface.

(A8) In the method denoted as (A7), the step of superimposing mayinclude superimposing, onto the model, an incision site for resectionsurgery, and projection of the local tissue onto the surface alongdirection from the local tissue to incision site.

(A9) In either of both of the methods denoted as (A7) and (A8), the stepof generating a visualization may further include indicating, in themodel, injection location and direction, relative to the surface, forinjection of a dye into the local tissue to indicate at least aperipheral portion of the local tissue so as to provide an assessmenttool for the resection, wherein the injection location and direction arepart of the surgical guidance cues.

(A10) In any of the methods denoted as (A7) through (A9), the step ofgenerating a visualization may further include communicating the model,with the at least one of the surgical guidance cues superimposedthereon, to a tracking stylus device to track position of the trackingstylus with respect to the patient and visualizing the position of thestylus in the model.

(A11) The method denoted as (A10) may further include transferring atleast one of the surgical cues to the surface using the stylus.

(A12) Any of the methods denoted as (A7) through (A9) may furtherinclude communicating the model, with the at least one of the surgicalguidance cues superimposed thereon, to an augmented reality device.

(A13) The method denoted as (A12) may further include, in the step ofgenerating a visualization (a) generating a volumetric model of aportion of the patient, the volumetric model including a model of thelocal tissue and a model of the surface, and (b) superimposing thesurgical guidance cues on the volumetric model, and also communicatingthe volumetric model, with the surgical guidance cues superimposedthereon, to an augmented reality device.

(A14) In any of the methods denoted as (A1) through (A6), the localtissue may be a tumor in a breast of the patient, the first image may beof the breast when the patient is in supine position, and the method mayfurther include manufacturing a locator form that fits the breast, whenin supine position, wherein the locator form includes featuresindicating at least a portion of the surgical guidance cues.

(A15) In the method denoted as (A14), the locator form may indicateincision site for resection surgery and projection of the tumor onto thesurface along direction from tumor to incision site.

(A16) In either of both of the methods denoted as (A14) and (A15), thestep of manufacturing may further include incorporating a raised needleport into the locator form to guide a needle to the tumor, such that dyeinjected by a syringe through the needle into the tumor provides anassessment tool or a surgical guidance tool for the resection, theraised needle port being one of the surgical guidance cues.

(A17) In any of the methods denoted as (A14) through (A16), the step ofmanufacturing may further include incorporating a raised needle portinto the locator form at incision site for resection surgery to guide ahook wire from the incision site to the tumor, the hook wire being oneof the surgical guidance cues.

(A18) Any of the methods denoted as (A1) through (A17) may furtherinclude (a) in the step of automatically determining, analyzing thefirst image to automatically determine an optimal incision site, and (b)in the step of generating a visualization, indicating the optimalincision site relative to the surface.

(A19) Any of the methods denoted as (A1) through (A17) may furtherinclude (a) receiving a user-defined incision site location, and (b) inthe step of automatically determining, determining the plurality ofsurgical guidance cues based upon the user-defined incision sitelocation and the first image.

(A20) Any of the methods denoted as (A1) through (A19) may furtherinclude performing a tissue resection procedure that utilizes thesurgical guidance cues to resect the local tissue.

(B1) A method for generating surgical guidance cues for resection of atumor from a breast may include (a) processing at least one supine imageof a breast to determine three-dimensional spatial properties of thetumor, and (b) based upon the three-dimensional spatial properties,generating a plurality of surgical guidance cues indicating thethree-dimensional spatial properties of the tumor with respect tosurface of the breast.

(B2) In the method denoted as (B1), the step of processing may includedetermining centroid position of the tumor and determining a vector fromthe centroid position to incision site for resection surgery.

(B3) In either or both of the methods denoted as (B1) and (B2), the stepof generating may include determining the incision site as a firstsurface point of surface of the breast closest to perimeter of thetumor.

(B4) In any of the methods denoted as (B1) through (B3), the step ofgenerating may include determining projection of the tumor onto thesurface of the breast along the vector.

(B5) The method denoted as (B4) may further include transferring theprojection onto the surface of the breast by indicating at least aplurality of margins of the projection on the surface of the breast.

(B6) In the method denoted as (B5), the margins may include cranialmargin, caudal margin, lateral margin, and medial margin of theprojection.

(B7) Any of the methods denoted as (B1) through (B6) may include (a) inthe step of processing, determining (i) an anterior point of theperimeter where the vector intersects the perimeter, and (ii) aposterior point of the perimeter where the perimeter intersects a linepassing through the centroid position, a first surface point of surfaceof the breast closest to perimeter of the tumor, and chest wallassociated with the breast, and (b) in the step of generating,determining (i) anterior margin of the tumor as distance between theanterior point and the first surface point, and (ii) posterior margin ofthe tumor as distance between the posterior point and the chest wallalong the line.

(B8) Any of the methods denoted as (B1) through (B7) may further includeperforming a tissue resection procedure that utilizes the surgicalguidance cues to resect the local tissue.

(C1) A system for generating surgical guidance cues for resection of alocal tissue from a patient may include (a) a location module forprocessing at least one image of the patient to determinethree-dimensional spatial properties of the local tissue, and (b) asurgical cue generator for generating the surgical guidance cues basedupon the three-dimensional spatial properties.

(C2) In the system denoted as (C1), the location module may include aposition calculator for determining centroid position of the localtissue from the image.

(C3) In either or both of the systems denoted as (C1) and (C2), thelocation module may further including a direction calculator fordetermining a vector from the centroid position to incision site forresection surgery.

(C4) In any of the systems denoted as (C1) through (C3), the surgicalcue generator may include an incision site calculator for determiningthe incision site as point on surface of the patient closest toperimeter of the local tissue.

(C5) Any of the systems denoted as (C1) through (C4) may further includean interface for receiving location information for the incision site.

(C6) In any of the systems denoted as (C1) through (C5), the surgicalcue generator may include a projection calculator for determiningprojection of the local tissue onto surface of the patient along avector from the centroid position to incision site for resectionsurgery.

(C7) In the system of claim 29, the location module further comprising aperimeter calculator for determining (a) an anterior point of perimeterof the local tissue where the vector intersects the perimeter, and (b) aposterior point of the perimeter where the perimeter intersects a linepassing through the centroid position and the incision site.

(C8) In any of the systems denoted as (C1) through (C7), the localtissue may be a tumor in a breast of the patient and the surgical cuegenerator may include a volumetric margin calculator for determining (a)anterior margin of the tumor as distance between the anterior point andincision site for resection surgery, and (b) posterior margin of thetumor as distance between the posterior point and the chest wall along aline passing through the centroid position and the incision site.

(C9) In any of the systems denoted as (C1) through (C8), the surgicalcue generator may include a projection margin calculator for determiningcranial, caudal, lateral, and medial margins of a projection of thelocal tissue onto surface of the patient along a vector from thecentroid position to incision site for resection surgery.

(C10) Any of the systems denoted as (C1) through (C9) may furtherinclude a visualization module for generating a model of the surface ofthe patient with the surgical guidance cues superimposed thereon.

(C11) In any of the systems denoted as (C1) through (C10), the localtissue may be a tumor in a breast of the patient and the at least oneimage may include at least one supine image of the breast.

(C12) In any of the systems denoted as (C1) through (C11), the at leastone image may include (a) a volumetric image of a portion of the patientincluding the local tissue and (b) a three-dimensional surface image ofsurface of the patient including location of incision site used forresection surgery, and the system may further include an imageregistration module for registering the volumetric image to the surfaceimage.

(C13) In the system denoted as (C12), the image registration module mayinclude a rigid body transformation module for rigidly translating orrigidly scaling the volumetric image to register the volumetric image tothe surface image.

(C14) In the system denoted as (C13), the local tissue may be a tumor ina breast of the patient, the volumetric image may be of the breast in afirst supine position, and the three-dimensional surface image may be ofthe breast in a second supine position.

(C15) In any of the systems denoted as (C12) through (C14), the imageregistration module may include a deformable transformation module fordeformably transforming the volumetric image to register the volumetricimage to the surface image.

(C16) In the system denoted as (C15), the local tissue may be a tumor ina breast of the patient, the volumetric image may be of the breast inprone position, and the three-dimensional surface image may be of thebreast in supine position.

(C17) In any of the systems denoted as (C1) through (C16), the at leastone image may include (a) a volumetric image of a portion of the patientincluding the local tissue and (b) a three-dimensional surface image ofsurface of the patient including location of incision site used forresection surgery, and the system may further include an optical surfaceimager for generating the three-dimensional surface image.

(C18) In the system denoted as (C17), the optical surface imager may bea structured-light imager for (a) illuminating the surface withstructured light, (b) capturing one or more reflection images of thestructured light reflected by the surface, and (c) processing the one ormore reflection images to determine the three-dimensional surface image.

(C19) In the system denoted as (C17), the optical surface imager may bea stereo camera.

(D1) A patient-specific locator form for guiding resection of a tumorfrom a breast may include (a) a first locator form surface matchingsurface of the breast, when in supine position, such that thepatient-specific locator form fits the breast when placed on the breastin supine position, and (b) a plurality of features indicating aplurality of surgical guidance cues, respectively.

(D2) The patient-specific locator form denoted (D1) may further includeone or more markers positioned to match one or more respective fiducialson the breast, such that the markers aid positioning of thepatient-specific locator form on the breast.

(D3) In either of both of the patient-specific locator forms denoted as(D1) and (D2), the plurality of features may include at least one raisedneedle port, each having shape and position to passively direct arespective needle to a respective location associated with the tumorwhen the patient-specific locator form is placed on the breast.

(D4) In the patient-specific locator form denoted as (D3), each of theat least one raised needle port may have an elongated cannulation thatdefines direction of the respective needle when inserted in theelongated cannulation.

(D5) In either of both of the patient-specific locator forms denoted as(D3) and (D4), one of the at least one raised needle port may be locatedat incision site for resection surgery.

(D6) In any of the patient-specific locator forms denoted as (D1)through (D4), the plurality of features may include a cut-out atincision site for resection surgery to provide access to the tumor whenthe patient-specific locator form is placed on the breast.

(D7) In the patient-specific locator form denoted as (D6), the cut-outmay have outline matching projection of the tumor onto the surface ofthe breast along direction from the tumor to the incision site.

(D8) Any of the patient-specific locator forms denoted as (D1) through(D7) may further include a material model of the tumor, and at least onerod for connecting the material model to the first locator form surfaceto indicate position of tumor relative to the surface of the breast.

(D9) In the patient-specific locator form denoted as (D8), the materialmodel of the tumor may include one or more features to indicate one ormore respective margins.

(D10) In either or both of the patient-specific locator forms denoted as(D8) and (D9), a portion of the patient-specific locator form associatedwith the first locator form surface may be at least partly transmissiveto light such that the material model is visible through the firstlocator form surface when the material model is connected to the firstlocator form surface via the at least one rod.

(E1) A method for manufacturing a patient-specific locator form forguiding resection of a tumor from a breast may include additivelymanufacturing, based upon three-dimensional image of surface of thebreast in supine position and a plurality of surgical guidance cues, athree-dimensional locator form that (a) has first locator form surfacematching the surface of the breast and (b) includes a plurality offeatures indicating the plurality of surgical guidance cues,respectively.

(E2) The method denoted as (E1) may further include capturing thethree-dimensional image of the surface when the breast is in supineposition.

(E3) In either or both of the methods denoted as (E1) and (E2), the stepof additively manufacturing may include additively manufacturing atleast one raised needle port in the locator form, wherein each of the atleast one raised needle ports has shape and position to passively directa needle to a respective location associated with the tumor when thelocator form is placed on the breast.

(E4) In the method denoted as (E3), the step of additively manufacturingat least one raised needle port may include additively manufacturing, inthe locator form at incision site for resection surgery, a needle porthaving shape to passively guide a hook wire from the incision site toanterior margin of the tumor.

(E5) In the method denoted as (E3), the step of additively manufacturingat least one raised needle port may include additively manufacturing, inthe locator form at incision site for resection surgery, a needle porthaving shape to passively guide a hook wire from the incision sitethrough center of the tumor to posterior surface of the tumor.

(E6) In either or both of the methods denoted as (E1) and (E2), the stepof additively manufacturing may include forming a cut-out at incisionsite for resection surgery to provide access to the tumor when thelocator form is placed on the breast.

(E7) Any of the methods denoted as (E1) through (E6) may further includedetermining the incision site as a first surface point of surface of thebreast closest to perimeter of the tumor, based upon thethree-dimensional surface image and a volumetric image of the breast.

(E8) Any of the methods denoted as (E1) through (E6) may further includereceiving position of a user-specified incision site.

(E9) In the method denoted as (E6), the step of forming a cut-out mayinclude forming the cut-out to match projection of tumor onto thesurface of the breast.

(E10) Any of the methods denoted as (E1) through (E9 may further includeincorporating a first feature in the locator form to indicate, when thelocator form is placed on the breast, projection of the tumor onto thesurface of the breast.

(E11) Either or both of the methods denoted as (E9) and (E10) mayfurther include determining the projection from the three-dimensionalsurface image and a volumetric image of the breast as projection oftumor onto the surface of the breast along direction from centroid oftumor to incision site for resection surgery.

(E12) Any of the methods denoted as (E1) through (E11) may furtherinclude analyzing the three-dimensional surface image and at least onevolumetric image of the breast to determine the plurality of firstsurgical guidance cues.

(E13) Any of the methods denoted as (E1) through (E12) may furtherinclude manufacturing a material model of the tumor indicating one ormore margins of the tumor.

(E14) The method denoted as (E13) may further include connecting thematerial model to the first locator form surface to indicate location ofthe tumor relative to the surface of the breast.

(F1) A patient-specific locator form for guiding resection of localtissue from a patient may include (a) a first locator form surfacematching surface of the patient near location of the local tissue, suchthat the patient-specific locator form fits the surface, and (b) aplurality of features indicating a plurality of surgical guidance cues,respectively.

(F2) In the patient-specific locator form denoted as (F1), the localtissue may be a tumor in a breast of the patient, and the first locatorform surface matching surface of the breast.

(F3) Either or both of the patient-specific locator forms denoted as(F1) and (F2) may further include one or more markers positioned tomatch one or more respective fiducials on the surface, such that themarkers aid positioning of the patient-specific locator form on thesurface.

(F4) In the patient-specific locator form denoted as (F3), the localtissue may be a tumor in a breast of the patient, the first locator formsurface may match surface of the breast, and the markers may include anopening positioned to match nipple of the breast.

(F5) In any of the patient-specific locator forms denoted as (F1)through (F4), the plurality of features may include at least one raisedneedle port, each having shape and position to passively direct arespective needle to a respective location associated with the localtissue when the patient-specific locator form is placed on the surface.

(F6) In the patient-specific locator form denoted as (F5), each of theat least one raised needle port may have an elongated cannulation thatdefines direction of the respective needle when inserted in theelongated cannulation.

(F7) In either or both of the patient-specific locator forms denoted as(F5) and (F6), one of the at least one raised needle port may be locatedat incision site for resection surgery.

(F8) In any of the patient-specific locator forms denoted as (F5)through (F7), one of the at least one raised needle port being mayinclude a removable spacer with length configured to (a) target oneposition within the patient when the spacer is in place, and (b) targetanother position within the patient when the spacer is removed.

(F9) In any of the patient-specific locator forms denoted as (F1)through (F8), the plurality of features may include a cut-out atincision site for tissue resection surgery to provide access to thelocal tissue when the patient-specific locator form is placed on thesurface.

(F10) In the patient-specific locator form denoted as (F9), the cut-outmay have outline matching projection of the local tissue onto thesurface along direction from the local tissue to the incision site.

(F11) Any of the patient-specific locator forms denoted as (F1) through(F10) may further include a material model of the local tissue and atleast one rod for connecting the material model to the first locatorform surface to indicate position of local tissue relative to thesurface.

(F12) In the patient-specific locator form denoted as (F11), thematerial model of the local tissue may include one or more features toindicate one or more respective margins of the local tissue.

(F13) In either or both of the patient-specific locator forms denoted as(F11) and (F12), a portion of the patient-specific locator formassociated with the first locator form surface may be at least partlytransmissive to light such that the material model is visible throughthe first locator form surface when the material model is connected tothe first locator form surface via the at least one rod.

Changes may be made in the above systems and methods without departingfrom the scope hereof. It should thus be noted that the matter containedin the above description and shown in the accompanying drawings shouldbe interpreted as illustrative and not in a limiting sense. Thefollowing claims are intended to cover generic and specific featuresdescribed herein, as well as all statements of the scope of the presentmethod and device, which, as a matter of language, might be said to falltherebetween.

What is claimed is:
 1. A method for manufacturing a locator formconfigured to guide resection of a tumor from a breast of a patient,comprising: capturing at least one first image of the breast of thepatient, including image of the tumor and image of at least a portion ofsurface of the breast of the patient; automatically determining, atleast in part based upon the first image, a plurality of surgicalguidance cues indicating three-dimensional spatial properties associatedwith the tumor; and generating a visualization of the surgical guidancecues relative to the surface, said generating a visualization including(a) generating a model of the surface and (b) superimposing at least asubset of the surgical guidance cues on the model of the surface, thesubset of the surgical guidance cues including (a) at least one site onthe surface of the breast and (b) at least one respective projection ofthe tumor onto the surface along direction from the tumor to therespective site.
 2. The method of claim 1, the step of generating atleast one first image comprising: capturing at least one second image ofthe breast of the patient when the breast of the patient is in a secondposition, the second image including a second-position image of thetumor and second-position image of at least a portion of the surface;capturing at least one surface image of the breast of the patient whenthe tumor and the portion of the surface is in a first positionassociated with the resection; and transforming the second image toregister the surface as captured in the second image to the surface ascaptured in the at least one surface image, to produce the first image.3. The method of claim 2, each of the first position and the secondposition being a supine position, the step of transforming comprisingproducing the first image by performing a rigid-body transformation ofthe second image to register the surface as captured in the second imageto the surface as captured in the at least one surface image.
 4. Themethod of claim 2, the first position being a supine position, thesecond position being a prone position, the step of transformingcomprising producing the first image by deformably transforming of thesecond image to register the surface as captured in the second image tothe surface as captured in the at least one surface image whileaccounting for tissue displacement of the breast between the proneposition and the supine position.
 5. The method of claim 1, furthercomprising: receiving a user-defined location of each of the at leastone site; and in the step of automatically determining, determining theplurality of surgical guidance cues based upon the first image and theuser-defined location of each of the at least one site.
 6. The method ofclaim 1, the surgical guidance cues including indication of location ofa position on perimeter of the tumor.
 7. The method of claim 6, thesubset of the surgical guidance cues including projection of theposition onto the surface along a line of sight.
 8. The method of claim1, further comprising: in the step of automatically determining,analyzing the first image to automatically determine an optimal incisionsite; and in the step of generating a visualization, indicating theoptimal incision site relative to the surface.
 9. The method of claim 1,further comprising: receiving a user-defined incision site location; andin the step of automatically determining, determining the plurality ofsurgical guidance cues based upon the user-defined incision sitelocation and the first image.
 10. The method of claim 1, capturingcomprising capturing the at least one first image of the breast with avolumetric imager.
 11. The method of claim 10, the volumetric imagerbeing one of a magnetic resonance imaging scanner, an ultrasound imagingdevice, a computerized tomography scanner, and a mammography X-Rayinstrument.
 12. A patient-specific locator form for guiding resection ofa tumor from the breast of a patient, comprising: a first locator-formsurface matching surface of the breast of the patient near location ofthe tumor, such that the patient-specific locator form fits the surfaceof the breast; and a plurality of features indicating a plurality ofsurgical guidance cues, respectively, the plurality of surgical guidancecues including (a) at least one site on the surface of the breast and(b) at least one respective projection of the tumor onto the surfacealong direction from the tumor to the respective site.
 13. Thepatient-specific locator form of claim 12, further comprising one ormore markers positioned to match one or more respective fiducials on thesurface of the breast, such that the markers aid positioning of thepatient-specific locator form on the surface of the breast.
 14. Thepatient-specific locator form of claim 13, the markers including anopening positioned to match nipple of the breast.
 15. Thepatient-specific locator form of claim 12, the plurality of featuresincluding at least one raised needle port, each of the at least oneraised needle port having shape and position to passively direct arespective needle to a respective location associated with the tumorwhen the patient-specific locator form is placed on the surface of thebreast.
 16. The patient-specific locator form of claim 12, the pluralityof features including at least one raised needle port, each of the atleast one raised needle port having shape and position to passivelydirect a respective needle to a respective margin of the tumor when thepatient-specific locator form is placed on the surface of the breast.17. The patient-specific locator form of claim 12, the plurality offeatures including at least one projection-indicating feature eachindicating a projection of the tumor along a respective line-of-sightfrom the surface of the breast.
 18. The patient-specific locator form ofclaim 17, forming each projection-indicating feature as one or morecutouts.
 19. The patient-specific locator form of claim 12, theplurality of features including at least one cut-out at an incision sitefor tissue resection surgery.